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INTRODUCTION 



Background of the Vocational Education Research Seminars 



sr. 



In 1961, the Research Committee of the American Vocational Association esta- 
blished as a major objective the development of research abilities of individuals 
engaged in, or otherwise interested in research. In this effort the Committee 
worked with research staff representing the several branches of vocational education 
in the U.S, Office of Education. 




The situation at that time, as the Committee assessed it, was characterized by 
scarcity of well- qualified individuals to do research, by little interest to enter 
research, and by very limited or no funds available to support research projects. 

It was recognized that researchers in vocational education needed greater administra- 
tive and financial support. The Research Committee chose as its major objective the 
development within administrators, supervisors, teacher-educators and potential 
researchers of (l) a positive acceptance of a need for research and (2) the desire 
and ability to engage in research activities. 

To achieve these objectives the Research Committee, in cooperation with personnel 
in the U.S. Office of Education and Purdue University, planned and conducted a one- 
week vocational education research seminar on research design in the spring of 1963. 
Attending were vocational educators from each branch of vocational education, 
representing every region in the United States. Dr. David Krathwohl of Michigan 
State University presented the major lectures. 

The result of the seminar exceeded expectations. Participants were stimulated 
to continue the study of research methodology and statistics, and to become more 
actively engaged in vocational education research. This seminar tended to develop 
a favorable climate toward research. Important to note is the fact that ni i of 
those invited did attend, with expenses paid personally or by their states. The 
apparent success of the Purdue seminar indicated a definite need for continuing the 
effort. 

In 1964 a series of three, one-week vocational education research seminars were 
initiated by the Research Committee and vocational education research personnel from 
the U.S. Office of Education, and were conducted by three universities. One seminar, 
at Pennsylvania State University, dealt with research design, with participants 
selected by the Research Committee. A second, with emphasis on the contribution of 
the social sciences to vocational education research, was conducted by the University 
of Illinois at Allerton House. Those attending the Purdue University seminar the 
previous year were invited to this one. A third seminar was held at Ohio State 
University for administrators of vocational education research. -The purpose was to 
generate more interest, and to increase involvement on the part of state vocational 
education directors and of the various university department heads. 

As in the previous year, the participants in the 1964 seminars were selected by 
the Research Committee and were invited by the host university. The participants 
(or their employers) paid for the travel, room, meals and registration fee; and the 
U.S. Office of Education provided a small amount of financial support to the 
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seminars. These seminars not only developed within the participants increased 
research knowledge and skill hut also increased their enthusiasm for research. This 
is evidenced by the fact that they subsequently became involved in research projects. 

A series of four, one-week seminars were conducted in 1965 following the pat- 
terns, for the most part, of the three conducted the previous year — with the addition 
of another for those who started with the Purdue seminar and who continued in the 
Illinois seminar in 1964. One major and significant contribution to these seminars 
was the financial support from P.L. 88-210, Section 4(c). The proposal developed by 
Rupert Evans (University of Illinois) for conducting the seminars, was approved by 
the U.S. Office of Education. These funds made it possible to obtain more and better 
consultants for the seminars as well as to reimburse participants for travel, meals 
and room. 

Pertinent information on the 1965 seminars follows: a beginning seminar on 

research design and analysis of variance was held May 3-7 at the University of 
Minnesota, with a new group of vocational educators selected. to attend. An inter- 
mediate seminar was conducted at the University of Nebraska, April ll-l6, on role 
of the social sciences in vocational education research. The participants in the 
previous year’s seminar on research design were invited to this session. An advanced 
seminar was held at Michigan State University, April 19-23, including most of the 
previous participants from the Purdue seminar (1963) and the University of Illinois 
(1964) . This seminar involved the evaluation of proposals, research methodology, 
and the contribution of psychology, sociology and economics to research in vocational 
education. A fourth seminar was conducted at Ohio State University, May 24-27, for 
administrative personnel in state offices, teacher education departments and the 
research units of state education departments or universities. This seminar focused 
upon the structure and function of an administrative unit in initiating, coordinating, 
conducting and administering research projects and programs. 

Through informal and semi-formal evaluation of the vocational education research 
seminars conducted in 1963, 1964 and 1965, it was concluded that the seminars had 
been successful and of benefit to the participants. Those present were very enthu- 
siastic about the results of the seminars. It was recognized that competent 
researchers could not be developed within one week, but that the achievement of 
these professional people was noteworthy — coupled with the fact that they became 
more active in research projects, which in turn stimulated further in-service 
education in research. 

With the passage of P.L. 88-210 and the availability of money for research in 
vocational education, many more people became interested in, involved in, and 
responsible for research. However, the number of qualified researchers had not 
materially increased. Vocational educators recognize the need to increase their 
knowledge and competence in research, and desire to do so. Many individuals, who 
had not previously attended a seminar, expressed to the American Vocational Associa- 
tion Research Committee and to others, their wish to attend one of the research 
seminars . 

The establishment of Research Coordinating Units in vocational education in 
state education departments or in universities, created the need for additional 
researchers. These units had the opportunity to provide leadership and coordination 
in research, and such service was needed. There was evidence, however, that person- 
nel so involved would need assistance in thinking through their roles, relationships 
and responsibilities. They also need help in developing essential competencies in 
the administration of research and developmental programs. 
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There were strong indications that well-trained research personnel were in 
short supply, and that those employed in universities and state departments of 
education would need to become more sophisticated in order to design and conduct 
research projects. An in-service program for staff members in research positions 
was thus urgently needed. Increased effectiveness and efficiency of individuals 
in the Research Coordinating Units and in other research positions, would have a 
positive effect on a wide range of research and developmental activities in the 
states. 



Purpose of Seminars 

The major objective of this project was to- develop further the research know- 
ledge, competencies, and interests of those engaged in research in the field of 
vocational education. The project continued, further developed and expanded the 
vocational education research seminars initiated and conducted by the Research 
Committee of the American Vocational Association in cooperation with the U.S. Office 
of Education and selected universities. Six problem-centered areas were identified 
in which vocational educators could develop research knowledge and abilities. The 
areas selected were: (l) development and coordination of research by state research 
coordinating units, (2) curriculum development, (3) tests and measurements, (4) 
occupational mobility and migration, (5) research design and (6) curriculum 
evaluation. 



Development of Seminar Project for 1966 

The Research Committee of AVA asked the administration at Cornell University 
in February 1965 to consider writing a proposal for funding and coordinating the 
research seminars in 1966. The reply was in the affirmative, with C. W. Hill named 
to direct the project. A meeting was called by David Bushnell in Washington, D.C. 
in May 1965 with selected representatives* from the U.S. Office of Education, AVA 
Research Committee and universities. The work at hand was to review past seminars 
and to recommend topics or problem areas for each seminar, as well as make fin al 
selection of host universities . 

The Project Director contacted the proposed host universities to ascertain 
whether or not they would sponsor a seminar, and if so, whom they would name as 
seminar directors. In cooperation with each seminar director the dates, objectives 
and content were selected for each seminar. The seminar director, with the assist- 
ance of a planning committee in each host university, was given the freedom to pl an 



* List of attendants: 

David Bushnell, Director, Occupational Research and Planning 
U.S. Office of Education 

Duane Nielsen, Director, Educational Resources Development Branch 
U.S. Office of Education 
Rupert Evans, University of Illinois 
H. M. Hamlin, North Carolina State University 
C. W. Hill, Cornell University 
Warren Meyer, University of Minnesota 
Elizabeth Ray, Pennsylvania State University 
William Schill, University of Illinois 
Robert E. Taylor, Ohio State University 



and to develop the seminar program, A quota of 40 participants was established for 
five of the seminars and 60 for the sixth. The six scheduled seminars are listed 
below: 



Problem Area of seminar 


Date 


Host University 


Development and Coordination of 
Research by State Research 
Coordinating Units 


January 30- 
February 4, 
1966 


Ohio State University 


Curriculum Development 


February 7- 
11, 1966 


University of Georgia 


Tests and Measurement in 
Research 


March 28- 
April, 1966 


Colorado State University 


Occupational Mobility and 
Migration 


April 17-22 
1966 


North Carolina State University 


Research Design 


May 2-6, 
1966 


Cornell University 


Curriculum Evaluation 


May 16-20, 
1966 


University of Illinois 



A proposal for six vocational education research seminars, as indicated above, 
was developed with the help of the seminar directors in each university, and was 
submitted to the U.S. Office of Education in June of 1965* Notification of the 
approval of the project came in September, with financial contract in December, 

This contract provided money to employ recognized authorities and specialists 
in education, vocational education, and in the disciplines contributing to research 
in vocational education, to serve as consultants in each of the seminars. Another 
major item in the budget was the reimbursement of travel, room and meal expense of 
invited participants . 



Selection of Participants 

One major change in conducting the 1966 series of seminars was in the method of 
selecting participants. A two-page seminar announcement listing problem area, host 
university, date, quota and content of seminars was developed along with an applica- 
tion form for distribution across the nation. It was mailed to state directors 
(with a letter requesting that it be distributed to staff), to participants in 
previous seminars , to directors of Research Coordinating Units , and to deans of 
schools of education. The September 1965 issue of the American Vocational Journal 
carried the announcement that those interested might obtain further information and 
an application form. The AVA Research Committee and other previous participants 
were asked to submit to the Project Director names of likely applicants who had not 
as yet attended a seminar. 

The applicant was given the opportunity to indicate his first choice, an 
alternate choice in case his first choice was oversubscribed, and a second-choice 
seminar at his own expense. 

General criteria for the selection of participants were: degrees held (ability 
to participate in, and benefit from, the seminar), time budgeted to research, 
involvement in research (completed projects, present projects and projects pending), 
and reason for attending first-choice seminar. 
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By November 17 > 1965, a total of 346 applications had been submitted within the 
deadline for the six seminars. An additional 36 applications were submitted later. 
As can he seen in Table 1, there were’ more applicants than could be accommodated in 
all the seminars except one. This one dealt with an area unfamiliar to vocational 
educators which may account for it's low registration. 



Table !. Seminar and Number of Applications for Each Seminar 



First , Alternate Second 
Seminar Quota choice to first choice 

choice 



Development and Coordination of Research 



by State Research Coordinating Units 


60 


58 


l6 


l4 


Curriculum Development 

» 


40 


82 


64 


9 


Tests and Measurement 


4o 


58 


44 


9 


Occupational Mobility and Migration 


4o 


23 


27 


4 


Research Design 


4o 


73 


57 


15 


Curriculum Evaluation 


4o 


52 


52 


15 



The information returned on the application form was coded and punched on IBM 
cards. The cards were sorted using the previously mentioned criteria, and then 
listed according to first, eELtemate-to-first-choice and second-choice seminars. 
This then produced a print-out by seminar choices, with individuals listed in the 
order that their qualifications met the criteria. 

A Selection Committee composed of the Research Committee and staff from the 
U.S. Office of Education met November 22 aind 23, 1965, to select participants for 
each of the seminars. Each member of the Selection Committee was provided with a 
print-out sheet. This sheet and the actual applications Were used in the selection 
of participants. Every selectee was assigned to his first choice except for a few 
who received an alternate or second-choice seminar. In addition to the applicants 
who were selected within the quota, two to five U.S. Office of Education staff 
members were designated to take part in each of the seminars. 

A list of those selected and an alternate list for each of the seminars was 
sent to the host university seminar director, who extended an invitation. 
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Information on Participants in the Six Seminars 



Some of the pertinent information was tabulated from the application forms of 
those selected to attend the seminars. These data give some background on location 
of employment, branch of vocational education, and individual positions. 

• The affiliations and positions of the participants invited from the states are 
shown in Table 2. A high percentage of those attending the Ohio State seminar were 
in the state departments of education. Many of the individuals in the Research 
Coordinating Units were located in state departments, and a few in universities. 

In the other five seminars a large majority of the participants were in universities 
affiliated with vocational education. 

The branches of vocational education with which the participants were identified 
are given in part two of Table 2. There were twice as many in agricultural and 
trade-industrial education branches as in home economics and distributive education. 

In the Ohio group 2 6 are listed under f, Other". Most of these were researchers, 
directors or administrators in vocational education, or from the Research Coordinating 
. Units . 



The positions in which the participants worked are presented in part three of 
Table 2. In all of the seminars 108 out of 239 reported themselves to be teacher 
educators. Many of these had some responsibilities for research. Fifty- four of the 
participants were the chirman or head of a department in a university or college. 
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Table 2. Affiliations and Positions of f Participants 
in the Six Vocational Education Research Seminars 



I. 



II. 



Affiliation or 
Position 



Institution or Agency 
Research Coordinating 
Unit 

Higher education 
(vocational) 

State education dept. 

(vocational) 

Higher education 
(other than voc.) 
State education dept. 

(other than voc. ) 
Public schools 

TOTALS: 

Branch of Vocational 
Education 
Agriculture 
Business 
Distributive 
Guidance and Counsel 
Health Occupations 
Home Economics 
Technical 



Number 



North Total 

Ohio . Colorado Carolina All 

State Georgia State State Cornell Illinois Seminars 



III. 



20 

8 

19 

2 

3 

0 

52 



3 

1 

1 

2 

5 

0 

7 



1 

33 

3 

1 

0 

1 

39 



8 

2 "<v v 

8 

1 

0 

5 

3 



0 

36 

4 

0 

0 

0 

40 



10 

3 
2 
2 
0 

4 

1 



4 

18 

7 

1 

0 

0 

30 



8 

2 

3 

0 

0 

0 

1 



1 

37 

2 

1 

0 

0 

4l 



7 

6 

4 

1 

0 

9 

2 



1 

30 

5 

1 

0 

0 

37 



10 

1 

5 

1 

1 

8 

0 



Trade and Industrial 


5 


9 


14 


7 


6 


7 


Other 


26 


3 


3 


7 


5 


3 


Non Vocational 


2 


0 


1 


2 


1 


1 


TOTALS: 


52 


39 


4o 


30 


4l 


37 


resent Position 
Administration 


20 


0 


l 


2 


0 


0 


Chairman/Head of Dept. 


15 


10 


8 


‘ 3 


9 


9 


Research 


6 


3 


6 


2 


2 


4 


Supervision > 


7 


2 


2 


6 


1 


4 


Teacher Education 
• (Professor) 


3 


22 


20 


15 


28 


20 


Teacher, Counselor, 
Instructor 


0 


2 


2 


2 


1 


0 


Other 


1 


0 


1 


0 


0 


0 


TOTALS: 


52 


39 


40 


30 


4i 


37 



27 

162 

4o 

6 

3 

1 

239 



46 
15 
23 

7 

6 

26 

14 

48 

47 
7 

239 



23 

54 

23 

22 

108 

7 

2 

239 
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Table 3 aihows the number of participants who had completed and/or were involved 
in research projects. For the 239 participants in the six seminars 66 had completed 
two or more projects; 63, one project; and 110 had completed none. Research projects 
underway were reported by l48 people, with 91 not so involved. 

The percentage of time allotted for research by 239 individuals attending the 
seminars is shown in part three of Table 3. One-fourth of these budgeted 19 
percent or less of their time for research and another almost equal number allotted 
20-29 percent for research. The next-largest group had practically all of their 
time devoted to research. 



Table 3# Research Involvement of Participants Attending 
the Six Vocational Education Research Seminars 



Kind of 
Involvement 



I. Research Projects 
Completed 
None 
One 

Two or More 

II. Research Projects 
Underway 
None 

One or More 

III. Percent of Time 

Allotted to Research 
0-9 
10 - 19 
20 - 29 
30 - 39 
40 - 49 
50 - 59 
60 - 69 
70 - 79 
80 - 89 
90 - 100 



Number 



North 



Ohio 

State Georgia 


Colorado 

State 


Carolina 

State 


27 


19 


15 


15 


10 


7 


13 


9 


15 


13 


12 


6 



22 


10 


l6 


ll 


30 


29 


24 


19 



6 


3 


5 


4 


4 


4 


3 


6 


9 


11 


9 


8 


1 


6 


3 


2 


1 


1 


0 


0 


5 


7 


9 


4 


1 


1 


0 


0 


4 


1 


1 


3 


3 


1 


0 


0 


18 


4 


10 


3 
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Total 

All 

Cornell Illinois Seminars 



19 15 110 

14 10 63 

8 12 66 



17 15 91 

24 22 148 



8 3 29 

7 5 29 

6 l4 57 

5 4 21 

2 0 4 

7 5 37 

1 1 4 

1 0 10 

0 2 6 

4 3 42 
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The degrees held, and age distribution of participants in the six seminars are 
given in Table 4, Seven of the 239 had a bachelor’s, 64 held master’s and 164 
doctoral degrees. A high percentage of the participants in the Colorado and Cornell 
seminars held doctoral, degrees. The age distribution shows that 105 or 44 percent 
of all the individuals were of ages 40-49. One-third of all the people were in the 
30-39 year interval. Seven reported their ages as 60 or more. 



Table 4. Degree Held and Age of Participants in the 
Six Vocational Education Research Seminars 



Number 



North Total 

Ohio Colorado Carolina All 

State Georgia State State Cornell Illinois Seminars 



I. Degree Held 



II. Age 



Bachelors 


3 


0 


0 , 


3 


0 


1 


7 


Masters 


20 


11 


6 


12 


7 


12 


68 


Doctorate 


29 


28 


34 


15 


34 


24 


164 


o\ 

CM 

1 

O 

CM 


1 


2 


3 


1 


2 


2 


11 


30-39 


15 


13 


15 


11 


l4 


8 


76 


40 - 49 


24 


16 


14 


l4 


18 


19 


105 


50 - 59 


10 


5 


6 


4 


6 


5 


36 


60 or more 


2 


3 


1 


0 


0 


1 


7 


Not reported 


- 


- 


1 


- 


1 


2 


4 
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Evaluation Procedure 



The purpose of the six seminars held in 1966 was: to develop further the 
research knowledge and competencies of individuals now engaged in, or soon to become 
engaged in, research activities in the field of vocational education. This state- 
ment of purpose led the investigators to design an evaluation emphasizing four 
areas: satisfactions, change in knowledge, research attitudes, and basic-applied 
research orientation. 

Since the evaluation was the responsibility of the project director rather 
than each seminar director, it was decided to use an approach which would be uniform 
in design for all seminars, yet would still allow for tailoring of the instruments 
to reflect the specific seminar content and purpose. 



Instruments Used 



Satisfactions . For measuring participant satisfactions concerning the seminar, 
a Likert-type instrument was adapted from a form developed by Welden (3). This 
form consisted of thirty statements to which each participant was to indicate his 
degree of agreement. An attached second-page asked five open-ended questions 
concerning the seminar and personal plans for applying the results. These were 
given to each participant on Thursday and were collected Friday morning. Neither 
page carried personal identification. (The participants were made aware of this 
fact. ) 

Knowledge . A pretest and posttest were developed for each of the seminars 
based upon the major content to be covered. The posttest consisted of twenty 
multiple-choice questions. These same twenty questions were divided into two groups 
for the pre-tests, i.e., the odd numbered questions grouped together comprised 
Form 1, while even numbered questions made up Form 2. The investigators developed 
the twenty questions used for the Ohio, Georgia, North Carolina and Illinois 
seminars on the basis of papers to be delivered at the seminar — which papers were 
available prior to the seminar. The director of the Colorado seminar devised the 
test to be given there but, followed the same general pattern of using one-half of • 
the posttest to make up each of the pretests. The two major consultants for the 
Cornell seminar (Krathwohl and MacEachem), each devised fourteen questions, with 
ten from each list selected by the investigators. 

Attitudes . The instrument used to study attitudes consisted of thirty research 
terms presented in the fom of an Osgood semantic differential. These research 
terms were selected by analyzing the overall proposal for all seminars and the 
purposes, objectives and proposed content of the six seminars. The terms chosen 
represented both specific terms related directly to one or two seminars, and general 
terms expected to be related to all seminars. 

This instrument consisted of thirty IBM cards each printed with a single 
concept and four bipolar seven point scales. These were selected from "The 
Measurement of Meaning" by Osgood and were all in the evaluative dimension. The 
cards were designed to allow participants to mark responses which were then read 
and punched by machine. 
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OPERATIONAL DEFINITION 
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0 
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valuable 


good 
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ft 

u 


ft 

u 


bad 



Figure 1. Example of card used in the assessment of attitudes. 



The thirty concepts were put in random order, and again the odd-numbered 
concepts were given as Form 1 of the pretest „ while the other fifteen were given as 
Form 2. The posttest comprised all thirty concepts. 

Basic vs. Applied Research Orientation . The fourth area studied consisted of 
two types of scales: (a) a modified basic vs. applied research instrument developed 
by Storer (2) and (b) a modified form adopted from "Organizing Educational Research" 
by Lazarsfeld and Sieber (l) which presented three questions to which responses 
could be placed on a five-point continuum. These two types were administered to 
all participants in the posttest. The pretest consisted Ox" one-half of the parti- 
cipants marking Storer* s instrument while the other half responded to the Lazarsfeld 
form. 



Design of the Administration of Evaluation 

Two forms for each of these three areas of evaluation (knowledge, attitudes, 
and research orientation) allowed a balanced design of eight possible combinations* 
The participants of each seminar were assigned one of these combinations by means 
of systematic sampling. 

The pretest packet containing one form of each of the three types were 
distributed on the Monday morning of each seminar. The posttest was administered 
on Friday morning. 



Analysis of Data 

The purpose of this evaluation was to analyze the effects of the six seminars 
in regards to satisfactions, knowledge, attitudes toward research, and applied vs. 
basic research orientation. The procedures used in analysis of the data are 
described below. The results are presented in connection with the summary dis- 
cussion of each seminar. 




Satisfactions. The responses to the satisfactions forms were analyzed by 
means of the Cornell Computing Center XTABS program. The median was then deter- 
mined by the graphic method. The medians and frequency distributions of responses 
to each statement are given for each seminar. 

Knowin g *- The responses to pretest and posttest questions were coded and 
punched on cards. A percentage score was then computed and punched on the same 
card for the ten-item pretest, the twenty-item posttest, and the ten items of the 
posttest which corresponded to the ten items given the participant in the pretest. 
These data were then analyzed to present the numbers of persons showing gains , 
losses, or no-change between pretest score and total posttest score, and between 
equivalent-items pretest and posttest score. 

Attitudes. Since the thirty research items were assigned at random to Form 1 
and Form 2 of the pretest, which in turn were assigned by systematic randomization 
to participants, it was assumed that the mean response to pretest items was an 
unbiased estimate of the group mean. 

The analysis therefore consists of a comparison of the estimated group means 
for pretest items with the actual group means for posttest items. These data are 
presented by means of a table of pretest and posttest mean responses per item, and 
a chart which illustrates the direction and magnitude of change from pretest to 
posttest. 

Basic vs. Applied Research Orientation . The two forms used to explore the 
orientations of participants toward basic and applied research were analyzed by 
comparing the responses given to each specific question at the time of pretest with 
the responses given to the same question, by the same participants four days later. 
The frequency distributions of responses to each question for both pretest and 
posttest, and the mean response value are given in tabular form. For comparison 
purposes the pretest and posttest means are also presented for all seminars (giving 
equal weight to each seminar). 
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SEMINAR PROGRAMS CONDUCTED 



Each host university selected or accepted a problem area around which a 
research program was planned and conducted* The director of each seminar was 
asked to use the assistance of a program-planning committee. Funds were available 
to engage outstanding consultants for the presentations. Insofar as was possible 
and consistent with the program plans, consultants were asked to prepare psipers in 
advance of the seminars for distribution to participants. The seminar directors 
at the Universities of Georgia and Illinois were very successful in obtaining copies 
of speeches before the seminars. In other seminars papers and varying techniques 
were used in presentations. 




A general observation reflected in the evaluation of the seminars by the 
participants, is that the consultants in the various disciplines such as sociology, 
economics, psychology, philosophy made good-to-excellent presentations. But the 
relevance to research varied greatly on a continuum from very applicable to of 
little use. This was apparent when research was presented from academic fields to 
illustrate designs for use in vocational education. The discussion became centered 
on the study — or in research itftdlf and its value per se--rather than upon its 
applicability to research in vocational education. 

A brief review of each seminar program is given in this section. Copies of 
the seminar programs and lists of participants are duplicated in the appendix. 

The data and discussion following the program are the results of the four types 
of instruments used in seminar evaluation. For optimum interpretation and under- 
standing the reader will find it necessary to study the Evaluation Procedures 
section appearing on pages 10-12. 

The tables and figures presenting the data on satisfactions and attitudes 
lend themselves to a much greater explanation and interpretation than it is feasible 
to present in this report. It is therefore suggested that study of these tables 
will prove informative. 
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Development and Coordination of Research by State Research 

Coordinating Units 

- Ohio - 



Program* -- . 

This seminar was planned and directed by Robert E. Taylor and Virgil Christensen 
at Ohio State University, January 31 - February k 9 1966. A special effort was made 
to develop a program for personnel in the Research Coordinating Units newly esta- 
blished within twenty-three states, and for personnel in other states which may 
establish units. 



In this seminar, major speeches and study were focused upon the role and 
policies of RCU staff, the structuring and organizing of RCU's, efficient and 
effective operations and management, selection of research problems and development 
of proposals, diffusion and dissemination of research and development information. 



The major speeches and consultants were: 



The Research and Development 
Concept - What Is There to be 
Done and Who Can Do It Best? 



Implications of Diffusion Research 
to Implemented Change in Education 



Improving Research in Vocational 
Education 



What Constitutes a Researchable 
Problem and a Respectable 
Proposal 

Putting Data Processing to Work 
in Vocational Education 

Putting PERT to Work in 
Research Coordinating Units 



Dissemination of Research and 
Development Information 



What Lies Ahead 



Ray Jongeward 
Director of Research 
Office of Public Instruction 
Olympia, Washington 

H. F. Lionberger 
Rural Sociology 
University of Missouri 

Alan Knox 
Teachers College 
Columbia University 

Egon Guba 

Professor of Education 
Ohio State University 

Loyal Joos 



- Desmond Cook 
-Professor of Education 
Ohio State University 

Harold Haswell 
Director, ERIC 
U.S. Office of Education 

David Bushnell, Director 
Adult and Vocational Research 
U.S. Office of Education 
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The seminar was dynamic and moving, with good balance, placement and timing of 
lectures? group interaction ; question periods after lectures; questions and reactions 
by participants; reactor panels to lectures; discussion groups; panels; discussion 
period with consultants; round-table discussions; stimulation sessions and workshop 
sessions* All of these methods and techniques made for varied, challenging 
interesting meetings involving the participants. * 

The proceedings of this seminar was published in the report "Research 
Development and Coordination - A Seminar Report" by Robert- Taylor and Virgil 
Christensen, Center for Vocational and Technical Education, The Ohio State 
University. 
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Evaluation 



Satisfactions . The data presented in Table 5 represent the frequency 
distribution and median for each of the thirty statements of the Satisfactions 
evaluation form. One-half of these statements were presented in the positive form 
with the others in the negative form. Rather than presenting a lengthy discussion 
of all these data the reader is advised to study the distribution of responses as 
well as the median for each of the statements. The following will therefore mention 
only a few of the highlights. 

The results of the Ohio seminar show agreement with all positive statements 
and disagreement with all negative statements. The median indicates that partici- 
pants agree most with statements l4, 29 and 7 and disagreed most with statements 
6 and 30. 



Knowledge . The results of the test on content presented in this seminar, 
resulted in 3 people attaining scores between 40-49JG, 8 between 50-59^ » 21 between 
60-69% 9 l4 between J0-J9% and 5 between 80—89%. Since no pretest* was given there 
is no indication of how much change resulted from this seminar. 



Because of heavy snow storms, it was not possible for the Project Director to 
administer the pre-evaluation forms. 
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Table 5. Median and Frequency Diatributlon 
for Degree of Agreement with Statements of Satisfaction 

- Ohio Seminar - 



Frequency Distribution 
Strongly Un- Dis- Strongly 

Median Is)* *?M* '“(s)' 4 *£)* “*(lT 



Statements 

In regard to this conference I feel that: 

1. The purposes of this program were clear to me 3.8 

2. The objectives of this program were not 

realistic 

3. Specific purposes made it easy to work 

efficiently 3.7 

U. The participants accepted the purposes of 

this program 3.6 

5. The objectives of this program were not the 

same as my objectives 1.7 

6. I didn't learn anything new 0.7 

7 • The material presented was valuable to me . . 4.1 

8. I could have learned as much by reading a 

took 0.8 

9. Possible solutions to my problems were 

considered o y 

10. The information presented was too elementary. .3 

11. The speakers really knew their subjects ... 3.6 

12. The discussion leaders were not well prepared 1.5 



13. I was stimulated to think objectively about 

the topics presented 3.6 

14. New acquaintances were made which will help 

in future research 4.2 

15. We worked together as a group ........ 3.8 

16. We did not relate theory to practice 1,6 

17. The sessions followed a logical pattern ... 3.6 

18. The schedule was too fixed. 1.5 

19. The group discussions were excellent. .... 3.4 

20. There was very little time for informal 

conversation 

21. I did not have an opportunity to express my 

ideas 1.3 

22. I really felt a part of this group 3.6 

23. MF time was well spent 3.7 

24. The program met my expectations 3.5 

25. I have no guide for future action 1.4 



26. Too much time was devoted to trivial matters. 1.3 

27. The information presented was too advanced. . 1.5 



28. The content presented was not applicable to 

research in vo~ed 

29. Seminars of this nature should be offered 

again in future years 4.2 

30. Seminars such as this will contribute little 

to vo~ed research 0,8 
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Attitudes * Since no pretest was administered to this group Table 6 shows 
only the posttest mean response per item for this seminar and the mean item 
response for all seminar posttests, giving equal weight to each seminar. 

The research terms receiving the highest evaluative mean responses in this 
seminar were: dissemination, funding, experimental research, data processing, and 
statement of problem. Other than the two negative terms, the only research term 
to receive a mean response of less than 5 was nominal scaling. 
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Table 6. Change in Attitudes toward Research Terms 

- Ohio Seminar ~ 



Mean 



Research Term 


Mean 

Posttest 


All Seminars 
Posttest 


Variance 


5.45 


5.315 


Non Parametric 


5.03 


4.905 


Sampling 


5.88 


5.828 


Dissemination 


6.05 


5.955 


Diffusion Research 


5.46 


5.003 


Applied Research 


5.65 


5.965 


Invalidity 


3.51 


3.153 


Funding 


6.16 


5.785 


Action Research 


5.73 


5.763 


Research Design 


5.82 


5.770 


Developmental Research 


5.72 


5.766 


Basic Research 


5.78 


5.703 


Experimental Research 


6.07 


6.040 


Operational Definition 


5.51 


5.608 


Nominal Scaling 


4.89 


4.956 


Research Proposal 


5.53 


5.620 


Measurement 


5.69 


5.751 


Hypotheses 


5.81 


5.688 


Research Findings 


5.66 


5.746 


Evaluation 


5.65 


5.771 


Contamination 


3.02 


2.958 


Inference 


5.23 


5.216 


Generalization 


5.59 


5.326 


Review of Literature 


5.40 


5.593 


Data Processing 


6.02 


5.753 


Randomization 


5.56 


5.596 


Theoretical Framework 


5.59. 


5.505 


Statistician 


5.71 


5.578 


Statement of Problem 


6.01 


5.851 


Reliability 


5.88 


5*871 
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Applied vs. Basic Research Orientation. The responses to the questions administered at the end 
of the seminar concerning participants’ orientation toward basic and applied research, are given in 
Tables 7 and 8. The first table gives the number of participants checking each of the five divisions 
on the continuum from applied to basic, and the mean response for this group and for all seminars. 

The second table indicates the number of persons checking each response to the five questions 
regarding research preference for this group and for all seminars. 

It can be seen from Table 7 that most participants indicated the prime concern of their present 
organization to be largely applied (X = 2.2). It is interesting to note that they prefer this concern 
to the slightly more basic (2.4) and their personal desires are still more basic (2.6). 

Table 8 shows that most would prefer to publish their research in popular publications (34), 
whereas there were l6 who would prefer publishing in scientific Journals. In making a choice between 
working on practical problems or contributing to development of a body of scientific knowledge, the 
majority (30) responded definitely for practical problems. The chance to serve people was of utmost 
importance, with only 11 persons not so indicating. * " 



Table 7. Frequency Distribution and Means of Responses to Lazarsfeld's 
Form of Applied vs. Basic Research Orientation 

- Ohio Seminar - 

A. Where would you locate the orientation or prime concern of the organization with which you are 
presently working? 



Frequency Distribution 



Mean 





Applied 


(1) (2) 


(3) 


(U) (5) 


Basic 


Ohio 


All Seminars 


Posttest 




13 25 


8 


4 l 




2.2 


2.15 


B. Where on the 


same continuum would you prefer the orientation of your organization to be located? 

Frequency Distribution Mean 




Applied 


(1) (2) 


(3) 


(U) (5) 


Basic 


Ohio 


All Seminars 


Posttest 




9 18 


20 


1 3 




2.4 


2.40 


C. Where on this 


continuum would you like to do research? 

Frequency Distribution 






Mean 




Applied 


(1) (2) 


(3) 


( k ) (5) 


Basic 


Ohio 


All Seminars 


Post test 




13 18 


7 


4 9 




2.6 


2.6l 
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Table 8. Means and Frequency Distribution of Responses to Storer's Scale 
of Basic vs. Applied Research Orientation 

- Ohio Seminar - 

A. If you had to choose between reporting your research in a popular publication where laymen would 
see it and perhaps use your findings, or reporting it in a scientific Journal, which would you 
prefer? 

Means 



Response 


Distribution 

Posttest 


Ohio 

Posttest 


All Seminars 
Posttest 


Definitely popular publication 
Somewhat popular publication 
I can't make up my mind 
Somewhat scientific journal 
Definitely scientific journal 


23 

11 

1 

10 

6 


2.3 


2.95 


If it ever came to a choice between working on the practiced problems of vocational education 
(problems important to the local schools), or contributing to the development of a body of 
scientific knowledge, which would you prefer to do? 








Means 


Response 


Distribution 
Posttest ‘ 


Ohio 

Posttest 


All Seminars 
Posttest 


Definitely practical problems 
Somewhat practical problems 
I can't make up my mind 
Somewhat scientific knowledge 
Definitely scientific knowledge 


30 

5 
h 

6 

6 


2.1 


2.51 


How important to you in a Job is the chance to serve people. 


i.e. , to help solve their problems? 


' 






Means 


Response 


Distribution 

Posttest 


Ohio 

Posttest 


All Seminars 
Posttest 


Of utmost importance 
Very important 
Somewhat important 
Hot very important 
Unimportant 


4o 

9 

1 

1 

0 


1.3 


n 

1.55 
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Curriculum Development 
- Georgia - 



Program 

This seminar was held at the University of Georgia, February 7-11* 1966, with 
Herschel Lester, Jr. serving as seminar director. 

The aim of the seminar was to help researchers in vocational education acquire 
better knowledge and more ability to undertake research in curricula development. 

The program was designed to analyze and criticize educational research reports, 
studies, and other literature directly related to designing and evaluating vocational 
technical curricula. The major emphasis was placed upon the participants 1 gaining 
a more complete understanding of the existing research methods believed to be useful 
in controlling curricula variables and in designing curricula development projects. 

To achieve this, the following presentations were given: 

Curriculum Studies - Duane Nielsen 

A Challenge Educational Resources 

Development Branch 
U.S. Office of Education 

Labor Market Analysis Robert E. Maritold 

and Projections U.S. Department of Labor 

The Interpretation of Norman S. Wood 

Economic Data Department of Economics 

University of Georgia 

Social Stratification Raymond Payne 

Department of Sociology 

i University of Georgia 

Relationships of Community Selz C. Mayo 

Environment to Vocational Department of Sociology 

Education Curriculum North Carolina State University 

Educational Psychology in 
the Curriculum 

Statistical Models in 
Curriculum Development 

The New Mathematics: A 

Pattern for Curriculum Reform 

* 

Curriculum Development , and 
Evaluation in English 

The Anthropology Curriculum 
Project as a Model for 
Curriculum Development: 

Practical Problems 



Joe Bledsoe 
College of Education 
University of Georgia 

Harry Anderson 

Center for Educational Stimulation 
University of Georgia 

Joseph R. Hooten, Jr. 

College of Education 
University of Georgia . 

Mary J. Tingle 
College of Education 
University of Georgia 

Marion Rice 
College of Education 
University of Georgia 



Panel Review of Current Curriculum Studies: 



Development and Evaluation of an 
Experimental Curriculum for the New 
Quincy, Massachusetts Vocational- 
Technical School 

An Experimental Evaluation of 
Approaches to Preparing High School 
Students for Agricultural Occupations 

Occupational Requirements in Office 
Occupations for School Learners 

Evaluation of Secondary School 
Programs to Prepare Students for 
Wage Earning in Occupations Related 
to Home Economics 

General Proposal Writing 



Edward J. Morrison 

American Institute of Research 



John Coster 

University of Nebraska 

Fred Cook 
Business Education 
Wayne State University 

Helen Nelson 

Department of Home Economics 
Education 
Cornell University 

Warren Findley 

Center for Educational Stimulation 
College of Education 
University of Georgia 



Panel - Formal Review of Prepared Curriculum Proposals by the University of 
Georgia Education staff: 

Kathryn Blake Karl King 

Charles Johnson James B. Konnecny 

Leonard Pikaart 



The principal method of presentation was a series of lectures by consultants, 
with a question period after each lecture. On Thursday afternoon and Friday morning, 
.a panel of five members of the University of Georgia faculty reviewed and criticized 
curricula-development proposals which had been submitted by the participants. This 
was a highlight in the seminar. 

The presentations for this seminar were compiled into a published report 
entitled, ’’Report of a National Vocational-Technical Education Seminar on Research 
and Curriculum Development" by Herschel T. Lester, Jr. , Editor and Seminar Director, 
Vocational Research, College of Education, University of Georgia, Athens, Georgia. 

It was distributed to participants and others in May 1966. 



Evaluation 



Satisfactions . The data presented in Table 9 represent the frequency distribution and median for 
each of the thirty statements of the Satisfactions evaluation form. One-half of these statements were 
presented in the positive form with the others in the negative form. Rather than presenting a lengthy 
discussion of all these data, it is advised that the reader study the distribution of responses as well 
as the median for each of the statements. The following will therefore mention only the highlights. 

The median responses for this seminar indicate agreement with all but two (3 and 24) of the 
positive statements, .and disagreement with all the negative statements. The positive statements 
resulting in greatest agreement are l4 and 29, whereas the negative statements receiving most dis- 
agreement are 6, 21, 27, and 30. 



Knowledge . The data presented in Table 10 gives the frequency distribution of changes in scores 
from pretest to posttest. The first row gives the frequency of change for scores computed on items 
which appeared on both the participants’ pretest and posttest. The second row gives the frequency of 
change from the participants’ pretest score to the score for all items on the posttest. Also given is 
the distribution of posttest scores. 

In this seminar 23 people increased their scores on the items appearing on both pretest and post- 
test. There were only 6 receiving lower scores, and 9 who showed no change. In the comparison of 
scores between pretest and total posttest there were 20 increasing their scores, while 11 remained the 
same, and 7 fell below their pretest level. 

The highest scores on the total item posttest were in the 80-89? range with most people achieving 
at least 60?. 



Table 10. Change in Content Knowledge Scores From Pretest to Posttest 

- Georgia Seminar - 







Number of Persons Showing 






Decreased 


No 




Increased Score by: 






Score 


Change 


10? 


fc*. 

O 

* 

O 

00 

t* 

O 

cu 


or more 


Change in scores for items 
on both pretest and posttest 


6 


9 


10 


3 5 5 




Change in scores from both 
pretests to all items on the 
posttest. 


7 


11 


8 


5 3 4 






Frequency Distribution of Posttest Scores 






Percentage 

score QI-19? 20-29? 


30-39? 4q-49? 


50-59? 


60-69? 


70-79? 80-89? 


90-99? 


Frequency 0 0 


6 5 


9 


13 


4 2 


0 



vUMmK** M 



irtrr i mu* 






ti,*k +t m&m 
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Table 9. Median and Frequency Distribution 
for Degree of Agreement with Statements of Satisfaction 

- Georgia Seminar - 



Frequency Distribution 



Strongly Un- Dis- Strongly 

Agree Agree decided agree Disagree 

Statements Median (5) (4) (3) (2) (l) 



In regard to this conference I feel that: 

1. The purposes of this program were clear to me 

2. The objectives of this program were not 

realistic 

3. Specific purposes made it easy to work 

efficiently 

4. The participants accepted the purposes of 

this program 

5. The objectives of this program were not the 

same as my objectives 

6. I didn't learn anything new 

7* The material presented was valuable to me . . 

8. I could have learned as much by reading a 

book. ,, 

9. Possible solutions to my problems were 

considered 

10. The information presented was too elementary. 

11. The speakers really knew their subjects . . . 

12. The discussion leaders were not well prepared 

13. I was stimulated to think objectively about 

the topics presented. 

14. New acquaintances were made which will help 

in future research 

15. We worked together as a group 

16. We did not relate theory to practice 

17. The sessions followed a logical patte. .... 

18. The schedule was too fixed. 

19. The group discussions were excellent 

20. There was very little time for informal 

conversation 

21. I did not have an opportunity to express my 

ideas 

22. I really felt a part of this group 

23. MF time was well spent 

24. The program met my expectations 

25. I have no guide for future action 

2b. Too much time was devoted to trivial matters. 

27. The information presented was too advanced. . 

28. The content presented was not applicable to 

research in vo-ed 

29 . Seminars of this nature should be offered 

again in the future . . . . . . 

30. Seminars such as this will contribute little 
to vo-ed research . . . . . . . . ... . 



3.3 


5 


19 


2 


10 


3 


1.8 


2 


8 


5 


18 


6 


2.8 


5 


13 


5 


11 


5 


3.2 


4 


20 


9 


6 


0 


1.9 


4 


12 


1 


l4 


8 


1.1 


2 


2 


4 


12 


19 


3.5 


12 


l4 


5 


7 


1 


1.5 


4 


6 


3 


12 


14 


3.3 


4 


22 


5 


8 


0 


1.9 


5 


9 


4 


15 


6 


3.6 


12 


16 


8 


2 


l 


1.5 


1 


3 


8 


15 


12 


.3.3 


8 


16 


3 


11 


1 


4.0 


19 


15 


3 


2 


0 


3.2 


5 


17 


4 


8 


5 


1.9 


5 


7 


5 


13 


9 


3.4 


8 


17 


3 


9 


2 


1.8 


7 


7 


1 


19 


5 


3.1 


5 


15 


5 


11 


3 


1.7 


2 


7 


3 


22 


5 


1.3 


0 


2 


3 


21 


13 


3.5 


8 


23 


4 


4 


0 


3.4 


9 


17 


4 


8 


1 


2.7 


5 


13 


3 


13 


5 


1.5 


0 


6 


i 


19 


11 


1.9 


2 


10 


5 


12 


10 


1.2 


0 


2 


0 


20 


17 


1.’5 


0 


5 


3 


21 


10 


4.0 


19 


13 


3 


4 


0 


1.2 


0 


4 


3 


15 


17 
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Attitudes* The data presented in Table 11 are the pretest and posttest means for this seminar 
and the means for all seminars, giving equal weight to each seminar. Figure 2 represents the direction 
and magnitude of change (in hundredths of a point) from pretest to posttest. The reader must consider 
the table and the chart together, as well as the relationship between the item means for this seminar 
and the item means for all the seminars. (It must also be noted that the direction of change from 
pretest to posttest response for the two negative items, i.e., invalidity and contamination* have been 
inverted so as to be consistent with the 28 positive items.) 

As was the case in presenting the data derived from the Satisfactions evaluation form, only the 
highlights of these data will be presented below. It is suggested that study of the data presented in 
both the table and chart will be most informative. 

An important factor to consider in analysis of the data presented here is the fact that the Georgia 
seminar recorded higher mean item responses on the pretest than any other seminar, for lb of the 30 
research terms. This then may indicate that the large degree of negative change from pretest to post- 
test is more a result of the statistical artifact of regression toward the mean, characteristic of 
test-retest results on the sematic differential, than of actual negative changes in attitude. A further 
indication that this may be the case is the observation that all but three of the research terms pretest 
means are above the average of pretest means for all seminars. 

None of the research terms received a mean posttest score of 6 or more. The terms receiving 
highest rating on the scales were applied research and action research. With the exception of the two 
negative terms, diffusion research was the only term to receive a posttest mean less than 5. 

The distribution of change can be seen to be almost completely negative. This (as mentioned above), 
may be primarily a result of regression toward the mean, in the judgment of investigators. The terms 
showing greatest negative change are: experimental research, measurement, theoretical framework, and 

statement of problem. The two terms showing greatest positive change sure invalidity and contamination. 
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Table 11 , Change in Attitudes toward Research Terms 
- Georgia Seminar - 



Mean 



Research Term 


Georgia 


All Seminars 


Pretest 


Posttest 


Pretest 


Posttest 


Variance 


5.1*8 


5.1*7 


5.272 


5.315 


Non Parametric 


5.19 


5.09 


1* .791* 


4.905 


Sampling 


5.90 


5.89 


5.71*8 


5.828 


Dissemination 


6.09 


5-91* 


6 . oil* 


5.955 


Diffusion Research 


4.8l 


1*.61* 


1 * . 76 I* 


5.003 


Applied Research 


6.09 


5.91* 


6.032 


5.965 


Invalidity 


3.00 


3.33 


3.01*1* 


3.153 


Funding 


6.00 


5.87 


5.1*50 


5.785 


Action Research 


6.22 


5.92 


5 . 671 * 


5.763 


Research Design 


5.73 


5.57 


5.790 


5.770 


Developmental Research 


6.01 


5.76 


5.756 


5.766 


Basic Research 


5.67 


5.50 


5.712 


5.703 


Experimental Research 


6.21 


5.67 


5.956 


6.040 


Operational Definition 


5.60 


5.66 


5.1*80 


5.608 


Nominal Scaling 


4.82 


5.10 


4.726 


4.956 


Research Proposal 


5.72 


5.63 


5.572 


5.620 


Measurement 


6.15 


5.58 


5.808 


5-751 


Hypotheses 


5.71 


5.61 


5.670 


5.688 


Research Findings 


5.93 


5.62 


5.800 


5.746 


Evaluation 


5.57 


5.63 


5.760 


5.771 


Cotamination 


2.58 


3.07 


2.91*1* 


2.958 


Inference 


5.61* 


5.31 


5.266 


5.216 


Generalization 


5.35 


5.25 


5.276 


5.326 


Review of Literature 


5.69 


5.1*8 


5.620 


5.593 


Data Processing 


5.91 


5.71 


5.746 


5.753 


Randomization 


5.1*6 


5.1*3 


5.354 


5.596 


Theoretical Framework 


6.05 


5.1*6 


5.734 


5.505 


Statistician ' 


5.52 


5.32 


5.462 


5.578 


Statement of Problem 


6.16 


5.73 


5.921* 


5.851 


Reliability 


5.81 


5.71 


5.742 


5.871 



Applied vs. Basic Research Orientation . The data in Tables 12 and 13 give the frequency of 
responses to each of three questions whicn were given to participants both in pretest and posttest. 

The tables also indicate the mean response to questions for this group as well as the mean response 
for all seminars, giving equal weights to seminars. 

The participants responding to this instrument indicated that their personal research preference 
is more basic than they would have their organisation undertake and also that the organization is 
concerned with work which is more applied than they would prefer. The data in Table 12 also show 
that only 5 out of 20 responded on the basic half of the continuum. 

Table 13 shows that in the pretest the participants were evenly distributed between publishing 
their research in popular and in scientific publications. The posttest, however, shows a slight shift 
toward preferring the scientific Journal. The choice between working on practical problems or con- 
tributing to the development of a body of scientific knowledge shows Just the opposite shift. The 
chance to serve people remained of great importance. 



Table 12. Means, Frequency Distribution of Responses to Lazarsfeld's 
Form of Applied vs. Basic Research Orientation 

- Georgia Seminar - 



A. Where would you locate the orientation or prime concern of the organization with which you are 
presently working? 





Response Frequency 




Basic 




Mean 


Applied (l) (2) 


(3) 


(b) 


(5) 


Georgia 


All Seminars 


Pretest 


7 8 


0 


5 


0 




2.2 


2.16 


Posttest 


5 7 


U 


h 


0 




2.1* 


2.15 


Where on 


the same continuum would you prefer the i 


orientation 


of your organization to be located? 




Response Frequency 








Mean 




Applied (l) (2) 


(3) 


(4) 


(5) 


Basic 


Georgia 


All Seminars 


Pretest 


2 8 


6 


3 


1 




2.7 


2.66 


Posttest 


3 7 


8 


2 


0 




2.5 


2.1*0 


Where on 


this continuum would you like to do 


research? 












Response Frequency 








' Mean 




Applied (1) (2) 


(3) 


(M 


(5) 


Basic 


Georgia 


All Seminars 


Pretest 


5 5 


5 


u 


1 




2.6 


2.54 


Post test 


k 7 


6 


3 


0 




3.0 


2.6l 
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Table 13, Means and Frequency Distribution of Responses to Storer's Scale 
of Basic vs. Applied Research Orientation 

- Georgia Seminar - 

A. If you had to choose between reporting your research in a popular publication where laymen would 
see it and perhaps use your findings, or reporting it in a scientific Journal, which would you 
prefer? 



Means 






Response • 


Distribution 
Pretest Fosttest 


Georgia 

Pretest Posttest 


All Seminars 
Pretest Posttest 


Definitely popular publication 


3 


It 


2.9 3.1 


2.96 2.95 


Somewhat popular publication 


6 


3 






I can't make up my mind 


1 


0 






Somewhat scientific Journal 


5 


9 






Definitely scientific Journal 


3 


2 






B. If it ever came to a chu j* between working on the practical problems of vocational education 


(problems important to the local schools), or 


contributing to the development of a body of 


scientific knowledge, which would you prefer to do? 












Means 




Distribution 


Georgia 


All Seminars 


Response 


Pretest 


Posttest 


Pretest Posttest 


Pretest Posttest 


Definitely practical problems 


k 


6 


2.9 2.5 


2.7k 2.51 


Somewhat practical problems 


It 


5 






I can't make up my mind 


1 


1 






Somewhat scientific knowledge 


8 


It 






Definitely scientific knowledge 


1. 


2 






C. How important to you in a Job is the 


chance to 


serve people 


, i.e., to help solve their problems? 








Means 




Distribution 


Georgia 


All Seminars 


Response 


Pretest 


Posttest 


Pretest Fosttest 


Pretest Posttest 


Of utmost importance 


13 


10 


l.k 1.6 


1.60 itj 


Very important 


3 


6 






8amewhat important 


2 


2 






Hot very Important 


0 


0 






Unimportant 


0 


0 
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Teats and Measurement 
- Colorado - 



Program 

This seminar was conducted at Colorado State University, March 28 - April 1, 
1966 under the direction of Douglas Sjogren, Seminar Director. A large percentage 
of the participants had attended previous years* seminars. Also, they had been 
and/or were involved in research projects. Interest in the problem area was very 
high-- as evidenced by the fact that only two invitees declined an invitation Just 
a few days before the seminar. Two others readily filled their place. 

Three major factors contributed to the success of the seminar. These factors 
were: (l) inclusion of adequate time for questions and discussion by the partici- 

pants concerning the major presentations, (2) the workshop on developing attitude 
scales and tests and (3) provision of time for participants to discuss many and 
varied topics and seminar presentations, to talk with consultants, and to complete 
workshop assignments. 



As may be seen below, there were few major speeches in the seminar; but there 
was adequate time for questions and discussion from the audience immediately 
following the presentation, and for later individual discussion with the consultant. 
The talks were well interspersed with workshop sessions. These were the major 
presentations: 



Tests 8 ' Measurements 



Test and Validity in 
Predictive Research 



Available Tests and Their Use 
in Research in Vocational 
Education 



Test Validity in Experimental 
and Curriculum Research 

Validity in Survey Research 



J. Stanley Ahmann 
Academic Vice President 
Colorado State University 

Garlie Forehand 
Department of Psychology 
Carnegie Institute of Technology 

Margaret Crawford 
Assistant Dean of Counseling 
and Guidance 

Los Angeles Trade-Technical College 

Robert Stake 
University of Illinois 

Alan Knox 
Teachers College 
Columbia University 



Dr. Charles 0. Neidt, assisted by Dr. C. Dean Miller, conducted a workshop (in 
the true sense of the word), beginning on Monday afternoon, and dealing with the 
development of attitude scales. First a review was given of the various techniques 
used and the theory behind them. The discussion and work focused upon the attitude 
scales, using the Thurstone and Likert Techniques. The participants were divided 
into four gzoups. Each was given the assignment of developing items and validating 
them. The workshop was continued on Thursday afternoon to review the past assign- 
ment and achievements and to set the stage for continued work. The workshop was 
completed on Friday morning. 
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No written report of the seminar content was planned. It was assumed that only 
the participants in the seminar were likely to benefit from the seminar. One of 
the values derived was the exchange of ideas, motivation and stimulation of interest 
in, and desire for, the proper use of tests and measurements. 
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Evaluation 



Satisfactions . The data presented in Table l4 represent the frequency distribution and median 
for each of the thirty statements of the Satisfactions evaluation form. One-half of these statements 
were presented in the positive form, with the others in the negative form. Rather than presenting a 
lengthy discussion of all these data the reader is advised to study the distribution of responses as 
well as the median for each of the statements The following will therefore point out only the 
highlights. 

The median responses of this seminar show agreement with all positive statements and disagreement 
with all negative statements. The statements resulting in greatest agreement are l4 and 29, while 
negative statements 6, 20, 21, and 30 showed greatest disagreement. 



Knowledge . The data presented in Table 13 gives the number of people and change in scores from 
pretest to posttest. The first row gives the frequency of change for scores computed on items which 
appeared on both the participants' pretest and posttest. The second row gives the frequency of 
change from the participants' pretest score to the score for all items on the posttest. Also given 
is the distribution of posttest percentage scores. 

This seminar appeared to have a rather difficult test, with no one receiving a total item 
posttest score greater than 39. Most were in the range between 40 and 49JS. 

There were only 6 persons showing gains in scores from pretest to same-item posttest, while 9 
showed no change, and 22 a loss. The pattern was repeated in a comparison of pretest and total 
posttest scores where 3 showed gains, l4 no change, aiid 1? a loss. 



Table 13. Change in Content Knowledge Scores from Pretest to Posttest 

- Colorado Seminar - 







Humber of Persons Showing 






Decreased 

Score 


No 

Change 


"1ST" 


Increased Score by: 

20* 30% 4of or more 


Change in scores for items 
on both pretest and posttest 


22 


9 


4 


2 


0 


0 


Change in scores from both 




l4 










pretests to all items on 
the posttest. 


_ 19 


4 


1 


0 


0 



Frequency Distribution of Posttest Scores 



Percentage 

score 


01-19* 


20-29* 


30-39* 


40-49* 


50-59* 60-69* 70-79* 80-89* 90-99* 


Frequency 


1 


2 


13 


20 


2 0 0 0 0 
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Table l4. Median and Frequency Distribution 
for Degree of Agreement with Statements of Satisfaction 

- Colorado Seminar - 



Frequency Distribution 



Statements 



In regard to this conference I feel that: 

1. The purposes of this program were clear to me 

2. The objectives of this program were not 

realistic 

3. Specific purposes made it easy to work 

efficiently 

4. The participants accepted the purposes of 

this program 

5. The objectives of this program were not the 

same as my objectives 

6. 1 didn't learn anything new 

7. The material presented was valuable to me . . 

8. I could have learned as nrich by reading a book 

9- Possible solutions to my problems were 

considered. . 

10. The information presented was too elementary. 

11. The speakers really knew their subjects . . . 

12. The discussion leaders were not well prepared 

13. I was stimulated to think objectively about 

the topics presented 

14. Hew acquaintances were made which will help 

In future research. • • • * 

15. We worked together as a group 

16 . We did not relate theory to practice 

17. The sessions followed a logical, pattern » . . 

18. The schedule was too fixed. * . 

19. The group discussions were excellent 

20. There was very little time for informal 

conversation «... 

21. I did not hove an opportunity to express ay 

ideas 

22. I really felt a part of this group 

23. My time was well spent 

e 

24. The program met ay expectations 

25. I have no guide for future action 

26. Too much time was devoted to trivial matters* 

27* The information presented was too advanced. . 

28. The content presented was not applicable to 

research in vo-ad . * ...... 

29. Seminars of this nature should be offered 

again in the future ....... 

30. Seminars such as this will contribute little 

to vo-ed research 



Median 


Strongly 

A^ree 


Agree 


Un- 

decided 

(3) 


Dis- 

agree 

(a) 


Strongly 

Disagree 

(1) 


3.6 


10 


24 


3 


1 


2 


1«5 


2 


2 


3 


23 


10 


3.4 


7 


22 


8 


2 


1 


3.5 


5 


27 


8 


0 


0 


1.7 


3 


6 


5 


20 


6 


1.1 


0 


0 


0 


21 


19 


3.6 


9 


25 


3 


1 


2 


1.4 


0 


3 


2 


25 


10 


3.3 


3 


24 


6 


7 


0 


1.2 


0 


3 


0 


22 


15 


3,6 


9 


24 


5 


0 


2 


1.4 


0 


2 


5 


20 


13 


3.7 


11 


27 


1 


1 


0 


4.1 


21 


15 


2 


2 


0 


3.8 


15 


24 


1 


0 


0 


1.5 


0 


4 


4 


23 


9 


3.4 


7 


22 


8 


2 


1 


1.4 


0 


0 


4 


' 25 


11 


3.2 


3 


21 


10 


6 


0 


1,1 


0 


1 


l 


20 


18 


LI 


0 


1 


0 


21 


18 


3.6 


7 


28 


3 


2 


0 


3.6 


10 


25 


4 


1 


0 


3.4 


6 


21 


6 


7 


0 


1.5 


0 


3 


5 


25 


7 


1.4 


1 


2 


4 


22 


11 


1.6 


1 


6 


3 


25 


5 


1.3 


1 


1 


3 


22 


13 


4.2 


24 


12 


4 


0 


0 


0.9 


0 


0 


4 


l4 


: 22 



it 



Attitudes. The data presented in Table 1 6 are the pretest and posttest means for this reminar, 
and the mean for, all seminars, giving equal weight to each seminar. Figure 3 represents the 
direction and magnitude of change (in hundredths of a point) from pretest to posttest. The leader 
must consider the table and the chart together as well as the relationship between the item means 
for this seminar and the item means over all the seminars. (It must also be noted that the direction 
of change from pretest to posttest response for the two negative items , i • e . , invalidity and 
contamination, have been inverted so as to be consistent with the 28 positive items.) 

As was the case in presenting the data derived from the Satisfactions evaluation form, only the 
highlights of these data will be presented below. It is suggested that study of the data presented 
in both the table and chart will be most informative. 

The research terms receiving a mean posttest response of 6 or more were sampling, applied 
research, research design, ejqierimental research, and reliability. The terms receiving less than 5 
on the posttest were non-parametric and diffusion research, in addition to the negative terms of 
contamination and invalidity. 

% - 

The direction of change for this seminar as shown in the chart is largely in this positive 
direction, with 11 terms showing a slight negative change. The term of greatest positive change 
was nominal scaling. 
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I Table 16 . 


Change in Attitudes toward Research 


Terms 






- Colorado Seminar 














Mean 






Colorado 


Ml Seminars 


Kesearcn Term 


Pretest 


Post test 


Pretest 


Posttest 


Variance 


5.23 


5.32 


5.272 


5315 


Ron Parametric 


*♦.97 


4.96 


4.794 


4.905 [ 


9 Sampling 


5.75 


6.01* 


5.71*8 


5.828 I 


a Dissemination 


5.92 


5.81* 


6.0il* 


5555 


9 Diffusion Research 


Ml 


4.96 


4.764 


5.003 


M Applied Research 


6.03 


6.17 


6.032 


5.965 


■ Invalidity 


2.66 


2.79 


3.01*1* 


3.153 


h Funding 


5.30 


5.56 


5.450 


5.785 


E Action Research 


5.«*5 


5.61* 


5.671* 


5.763 


H Research Design 


6.07 


6.ll» 


5.790 


5.770 


1 Developmental Research 


5.66 


5.86 


5.756 


5.766 


9 Basic Research 


5.90 


5.79 


5.712 


5.703 K 


■ Experimental Research 


5.97 


6.18 


5.956 


6.0l»0 p 


1 Operational Definition 


5.69 


5.60 


5.1*80 


5. 608 6 


1 Nominal Scaling 


Mi 


5.16 


4.726 


4.956 iv 


■ Research Proposal 


5.61* 


5.70 


5.572 


5.620 p 


1 Measurement 


5.67 


5.97 


5.808 


5.751 V 


| hypotheses 


5.92 


5.59 


5.670 


5.688 1 


I Research Findings 


5.79 


5.94 


5.800 


5.71*6 P 


1 Evaluation 


5.71 


5.91* 


5.760 


5*771 If 


1 Contamination 


2.71 


2.61* 


2.91*1* 


2.958 K 


1 Inference 


5.1*1* 


5.31 


5.266 


5.216 H 


S Generalisation 


5.30 


5.1*5 


5.276 


5.326 H 


1 Review of Literature 


5.60 


5.69 


5.620 


5.593 Bj 


1 Data Processing 


5.85 


5.77 


5.71*6 


5.753 Sr 


I Randomisation 


5.67 


5.69 


5.351* 


5.596 ■ 


1 Theoretical Framework 


5.80 


5.68 


5.731* 


5.505 H 


9 Statistician 


5.79 


5.78 


5.1*62 , 


5.578 ■ 


1 Statement of Problem 


5.90 


5.88 


5.921* 


5,851 ■ 


i v Reliability 

1 ’ l 


5.89 


6.07 


5.7l*2 


5.871 R 


I ^ i ■ 






't* 


1 
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Applied vs . Basic Research Orientation . The data in Tables 17 and 18 give the frequency of 
responses to each of three questions which were given to participants both in pretest and postttest. 
The tables also indicate the mean response per question for this group, as well as the mean response 
for all seminars, giving equal weight to seminars. 

Table 17 indicates little change in any of the three questions concerning orientation of the 
organizations or the participants. The results again show, however, that the participants prefer 
their organization to be concerned with more basic work than at present and also that the participants 
personally prefer an even more basic orientation. 

The data in Table 18 indicates a slight shift toward the more scientific approach in questions 
one and two. The responses to question 3, however, show a shift to more importance being given to 
the chance to serve people. 



Table 17. Means and Frequency Distribution of Responses to Lazarsf eld's 
Form of Applied vs, Basic Research Orientation 

- Colorado Seminar - 



A. Where would you locate the orientation or prime concern of the organization with which you 
presently working? 



Mean 





Applied (l) 


(2) 


(3) 


(U) 


(5) 


Basic 


Colorado 


All Seminars 


Pretest 


3 


11 


2 


1 


1 




2.2 


. i" 

2.16 


Posttest 


1 


11 


5 


1 


0 




2.4 


2.15 


Where on 


the same continuum would you prefer the orientation < 


of your organization to be located? 






Response Frequency 








Mean 




Applied (l) 


(2) 


(3) 


(4) 


(5) 


Basic 


Colorado 


All Seminars 


Pretest 


2 


7 


. 6 


2 


1 




2.6 


2.66 


Posttest 


2 


9 


6 


1 


0 




2.3 


2.40 


Where on 


this continuum would you like 


! to do 


research? 




- 








- 


Response Frequency 








Mean 




Applied (l) 


(2) 


(3) 


(4) 


(5) 


Basic 


Colorado 


All Seminars 


Pretest 


1 


*8 


4 


2 


3 




2.9 


2.54 


Posttest 


1 


8 


5 


4 


0 




2.7 


2.6l 



1 i 






1 
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Table 18. Means and Frequency Distributions of Responses to Storer's Scale 
of Basic vs. Applied Research Orientation 

- Colorado Seminar - 



A. If you had to choose between reporting your research in a popular publication where laymen would 
see it and perhaps use your findings, or reporting it in a scientific journal, which would you 
prefer? 









Means 


Response 


Distribution 


Colorado 


All Seminars 


Pretest 


Posttest 


Pretest Posttest 


Pretest Posttest 


Definitely popular publication 


6 


3 


2.8 3.2 


2.96 2.95 


Somewhat popular publication 


5 


5 


I can't make up my mind 


0 


0 






Somewhat scientific journal 


5 


10 






Definitely scientific journal 


k 


2 







B. If it ever came to a choice between working on the practical problems of vocational education 
(problems important to the local schools ) , or contributing to the development of a body of 
scientific knowledge, which would you prefer to do? 



« 

Means 

Distribution Colorado All S»nytwars 

Response Pretest Posttest Pretest Posttest Pretest Posttest 



Definitely practical problems 6 
Somewhat practical problems 2 
I can't make up my mind 0 
Somewhat scientific knowledge 7 
Definitely scientific knowledge 5 



6 2.8 3.0 2.7k 2.51 

2 
0 

10 

2 



C. How important to you in a Job is the chance to serve people, i.e., to help solve their problems? 



Response 



Distribution 
Pretest Posttest 



Means 



Colorado 

Pretest Posttest 



All Seminars 
Pretest Posttest 



Of utmost importance 
Very important 
Somewhat important 
Not very important 
Unimportant 



7 

9 

3 

1 

0 



9 

7 

k 

0 

0 



1.9 



1.8 



1.60 



1.55 



A 
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Occupational Mobility and Migration 
- North Carolina - 



Program 

jjiig research seminar was conducted at North Carolina State University , 

April 18-22, 1966 with Harry Beard serving as Director for the host university. 

The planning committee, with the help of three consultants, met several weeks 
before the seminar to plan and develop objectives and program content, and to 
coordinate the various presentations. Topics particularly pertinent to vocational 
educators were? (l) problems and decisions facing educational policy makers, program 
planners, and administrators because of occupational mobility and migration, (2) 
a conceptual framework for studying socio-economic mobility, (3) the status of 
research, (4) research rationales, designs, and methodology for the problem area, 

( 5 ) areas and researchable problems, and (6) resources available to conduct 
research and disseminate findings. 

The stage was set for the seminar by Rupert Evans' speech entitled, "The Need 
for Research and Its Utilization in Local, Regional and State Systems of Education. 
H. M. Hamlin followed with his reactions and support of Evans' talk. Attendants 
were further prepared for the study of occupational mobility and migration by a 
paper entitled, "Manpower Adjustments and Occupational Education," by E. Walton 
Jones, North Carolina State University. In the preparation of the paper, Jones 
compiled information on: manpower adjustments to economic growth, demand for 
manpower, the changing location of jobs, the supply of manpower, long-run changes, 
the mobility of the work force and the factors influencing it , gaps in mobility , 
manpower research, and the role of occupational education. Next, a series of 
papers was presented on topics by consultants as listed below: 



Manpower Supply in uhe 
United States 



Manpower Requirements by 
Industry and Occupations 



Howard Rosen 

Office of Manpower Policy, 
Evaluation and Research 
U.S. Department of Labor 

Sol Swerdloff 

Bureau of Labor Statistics 

U.S. Department of Labor 

Everett S. Lee 
Population Studies Center 
University of Pennsylvania 

William P. Kuvlesky 
Professor of Rural Sociology 
Texas A and M University 



k 



jjs 
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The next major activity of the seminar was to divide the participants into five 
workshop groups. A group was assigned to each of the following: (l) Manpower 

supply, (2) manpower requirements, (3) migration, (4) social-psychological dimen- 
sions of occupational mobility, and (5) socio-economic mobility. Each session was 
advised by the consuxtant who had given the presentation on the topic. On the last 
day a report of the deliberations of each group was presented before all the j 
participants. The Seminar Director assembled the five interest-group reports 
into the 38-page mimeograph summary entitled, "Seminar on Occupational Mobility 
and Migration - Report of Interest Groups." This was distributed to participants 
in early June 1966. A final report for this seminar entitled, "Occupational 
Mobility and Migration - Proceedings of a Vocational Education Research Seminar" 
was printed and distributed in August 1966. 

As indicated earlier, a comparatively small number of vocational educators 
originally expressed interest in occupational mobility and migration. The interest, 
expressions and participation of those in the seminar, however, wouljl indicate 
this to be an area of vital concern to vocational educators. Further, it may be 
stated that many vocational education leaders and those in research positions should 
pursue the subject because of its profound implications upon vocational education. 
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Evaluation 

♦ 

Satisfactions . The data presented in Table 19 represent the frequency distribution and median 
for each of the thirty statements of the Satisfactions evaluation form. One-half of these statements 
were presented in the positive form, with the others in the negative form. Rather than presenting a 
lengthy discussion of all these data it is advised that the reader study the distribution of responses 
as well as the median for each of the statements. The following Will therefore mention only the 
major points of interest. 

In this seminar, agreement was indicated for each positive statement and disagreement for each 
negative statement. The greatest agreement was given to positive statements 7, 23 and 29. The 
greatest disagreement resulted concerning negative statements 6, 8 and 30. 



Knowledge . The data presented in Table 20 gives the distribution of change in scores from 
pretest to posttest. The first row gives the frequency of change for scores computed on items 
which appeared on both the participants' pretest and posttest. The second row gives the frequency 
of change from the participants' pretest score to the score for all items on the posttest. Also 
appearing in the table is the distribution of posttest scores. 

There were 22 participants in this seminar who showed a gain in scores between pretest and the 
same-item posttest. Five showed a loss. The distribution of change was similar between scores in 
pretest and total-item posttest, with 20 gaining, 6 remaining the same, and 6 showing a loss. 

There was one participant who scored in the 90's for the total-item posttest, with a total of 
21 receiving at least a 60. 



Table 20. Change in Content Knowledge Scores from Pretest to Posttest 

- North Carolina Seminar - 







Number of Persons Showing 






Decreased 


No 




Increased Score by: 




Score 


Change 


10# 


ro 

0 


0 

cn 


40% or more 


Change in scores for items 
on both pretest and posttest 


. 5 


5 


9 


5 


6 


2 


Change in scores from both 
pretests to all items on 
the posttest. 


6 


6 


11 


2 


7 


0 



Frequency Distribution of Posttest Scores 

Percentage 

score 01-19# 20-29# 30-39# Uo-l*9# 50-59# 60-69# 70-79# 80-89# 90-99# 

Frequency 6 6 11 2 7 0 



Table 19, Median and Frequency Distribution 
for Degree of Agreement with Statements of Satisfaction 

- North Carolina Seminar - 



Frequency Distribution 



Strongly Un- Dis- Strongly 

Agree Agree decided agree Disagree 

Statem ents Median (5) (It) (3) (2) (l) 

In regard to this conference I feel that: 



1. The purposes of this program were clear to me. 


3.5 


5 


23 


2 


2 


0 


2. The objectives of this program were not 


realistic 


1.6 


l 


6 


1 


17 


7 


3. Specific purposes made it easy to work 


efficiently 


3.0 


4 


12 


10 


6 


0 


U. The participants accepted the purposes of this 


program 


3.5 


6 


20 


4 


2 


0 


5. The objectives of this program were not the 


same as my objectives 


1.8 


2 


6 


3 


19 


2 


6. I didn't learn anything new 


0.7 


0 


0 


0 


7 


25 


7. The material presented was valuable to me. . . 


4.1 


18 


13 


0 


1 


0 


8. I could have learned as much by reading a book 


0.8 


0 


0 


0 


12 


20 


9. Possible solutions to my problems were 

considered 


3.3 


4 


16 


7 


3 


2 


10. The information presented was too elementary . 


1.2 


0 


0 


0 


20 


12 


11. The speakers really knew their subjects. . . . 


3.9 


l4 


16 


1 


1 


0 


12. The discussion leaders were not well prepared. 


1.7 


3 


7 


2 


10 


10 


13. I was stimulated to think objectively about 


the topics presented 


3.6 


9 


17 


4 


2 


0 


14. New acquaintances were made which will help in 


future research 


3.9 


14 


15 


3 


0 


0 


15. We worked together as a group 


3.8 


12 


17 


2 


1 


0 


16. We did not relate theory to practice 


1.8 


3 


6 


4 


12 


7 


17. The sessions followed a logical pattern. . . . 


3.6 


7 


20 


3 


1 


1 


18. The schedule was too fixed 


1.6 


2 


5 


2 


18 


5 


19. The group discussions were excellent 


3.6 


10 


l4 


4 


4 


0 


20. There was very little time for informal 


conversation 


1.4 


0 


1 


l 


23 


7 


21. I did not have an opportunity to express my 


ideas .... 


1.3 


2 


1 


l 


16 


12 


22. I really felt a part of this group 


3.7 


10 


20 


2 


0 


0 


23. My time was well spent 


4.1 


17 


13 


1 


1 


0 


24. The program met my expectations 


3.5 


7 ' 


18 


4 


2 


1 


25. I have no guide for future action 


1.2 


0 


1 


2 


16 


13 


26. Too much time was devoted to trivial matters . 


1.2 


0 


1 


0 


17 


14 


27. The information presented was too advanced . . 


1.4 


0 


5 


0 


17 


10 


28. The content presented was not applicable to 


research in vo-ed 


1.5 


0 


1 


3 


21 


7 


29. Seminars of this nature should be offered 


again in future years 


4.4 


26 


6 


0 


0 


0 


30. Seminars such as this will contribute little 


to vo-ed research 


0.7 


0 


0 


0 


9 


23 




Attitudes, The data presented in Table 21 are the pretest and posstest means for this ^seminar 
arid the means for all seminars, giving equal weight to each seminar. Figure 4 represents the 
direction and magnitude of change, in hundredths of a point, from pretest to posttest. The reader 
must consider the table and the chart together as well as the relationship between the item means 
for this seminar and the item means for all the seminars. (It must also noted that the direction 
of change from pretest to posttest response for the two negative items, i.e., invalidity and 
contamination, have been inverted so as to be consistent with the 28 positive items.) 

As was the case in presenting the data derived from the Satisfactions evaluation form, only the 
highlights of these data will be presented below. It is suggested that study of the data presented 
in both the table and chart will be most informative. 

This s emi nar had lH research terras on the pretest which received an average response lower than 
any other seminar. Of the thirty response means, 25 were lower than the mean computed for all 
seminars. This might indicate a regression toward the mean in a manner opposite that of the Georgia 

seminar. 

The research terms receiving a mean posttest response of more than 6 were dissemination, 
applied research, and action research. The terms with a mean of less than 5 on the posttest were 
non-parametric , nominal scaling, and generalization, in addition to the two negative terms of 
invalidity and contamination. 

The change as shown in the chart is in the positive direction, but as mentioned above this may 
be attributable largely to the statistical artifact of regression toward the mean. The terms 
receiving greatest positive shift are diffusion research and funding. The term theoretical frame- 
work exhibited the greatest negative change. 
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Table 21. 



Change in Attitudes toward Research Terms 



- North Carolina Seminar - 



Mean 



Research Term 


North 


Carolina 


All Seminars 


Pretest 


Posttest 


Pretest 


Posttest 


Variance 


5.07 


5.22 


5.272 


5.315 


Non Parametric 


4.77 


4.72 


4.794 


4.905 


Sampling 

* 


5.73 


5.68 


5.748 


5.828 


Dissemination 


5.76 


6.09 


6.014 


5.955 


Diffusion Research 


4.88 


5.39 


4.764 


5.003 


Applied Research 


5.78 


6.07 


6.032 


5.965 


Invalidity 


2.91 


2.95 


3.044 


3.153 


Funding 


5.07 


5.68 


5.450 


5.785 


Action Research 


5.81 


6.02 

r 


5.674 


5.763 


Research Design 


5.21 


5.53 


5.790 


5.770 


Developmental Research 


5.72 


5.73 


5.756 


5.766 


Basic Research 


5.48 


5.49 


5.712 


5.703 


Experimented Research 


5.85 


5.77 


5.956 


6.o4o 


Operational Definition 


5.23 


5.58 


5.480 


5.608 


Nominal Scaling 


4.91 


4.95 


4.726 


4.956 


Research Propos'd 


5.23 


5.54 


5.572 


5.620 


Measurement 


5.78 


5.63 


5.808 


5.751 


Hypotheses 


5.64 


5.50 


5.670 


5.688 


Research Findings 


5.72 


5.54 


5.800 


5.746 


Evaluation 


5.55 


5.67 


5.760 


5-771 


Contamination 


3.07 


3.01 


2.944 


2.958 


Inference 


5.10 


5.00 


5.266 


5.216 


Generalization 


4.94 


4.95 


5.276 


5.326 


Review of Literature 


5.14 


5.40 


5.620 


5.593 


Data Processing 


5.72 


5.72 


5.746 


5.753 


Randomization 


5.05 


5.37 


5.354 


5.596 


Theoretical Framework 


5.60 


5.16 


5.734 


5.505 


Statistician 


5.19 


5.48 


5.462 


5.578 


Statement of Problem 


5.95 


5.74 


5.924 


5.851 


Reliability 


5.53 


5.79 


5.742 


5.871 
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Applied vs. Basic Research Orientation . The data in Tables 22 and 23 give the frequency of 
responses to each of three questions which were given to participants both in pretest and posttest. 
The tables also indicate the mean response per question for this group as well as the mean response 
for all seminars,, giving equal weight to each seminar. 

The only question showing change of any appreciable magnitude from pretest to posttest in 
Table 22 is question 2 concerning the preferred orientation of the participants organisation. 

Here the shift was away from basic, and toward applied. The posttest means indicate again that 
the participants prefer to do research which is more basic than they would have their organisations 
undertake, and also that their organisations presently are concerned with work which is more 
applied than they would prefer. 

According to the data shown in Table 23 three participants changed from reporting their 
research in a definitely popular publication to a somewhat more scientific Journal. The shift was 
in the opposite direction for the problems they would prefer to work on, while very little shift 
occurred in their desire to serve people. 



Table 22. Means and Frequency Distribution of Responses to Lazarsfeld’s 
Fora Applied vs. Basic Research Orientation 

- North Carolina Seminar - 



A. Where would you locate the orientation or prime concern of the organization with which you are 
presently working? 



Response Frequency Mean 





Applied (1) 


(2) 


(3) 


(4) 


(5) 


Basic 


N. Carolina 


All Seminars 


Pretest 


4 


6 


3 


1 


0 




1.9 


2.16 


Posttest 


5 


7 


2 


0 


0 




1.8 


2.15 


B. Where on 


the same continuum would you prefer the 


orientation 


of your organization to be located? 






Response Frequency 








Mean 




Applied (l) 


(2) 


(3) 


(4) 


(5) 


Basic 


N. Carolina 


All Seminars 


Pretest 


1 


6 


5 


1 


1 




2.6 


2.66 


Posttest 


2 


9 


3 


0 


0 




2.1 


2.40 


C. Where on 


this continuum would you like to 


do research? 














Response Frequency 








Mean 




Applied (l) 


(2) 


(3) 


(4) 


(5) 


Basic 


N. Carolina 


All Seminars 


Pretest 


3 


4 


6 


0 


1 




2.4 


2.54 


Posttest 


2 


7 


4 


1 


0 




2.3 


2.61 



rir^>^T»n 
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Table 23. Means and Frequency Distributions of Responses to Sfcorer's Scale 
of Basic vs. Applied Research Orientation 

- North Carolina Seminar - 



A. If you had to choose between reporting your research in a popular publication where laymen would 
see it and perhaps use your findings, or reporting it in a scientific Journal, which would you 
prefer? 

Means 





Distribution 


North Carolina 


All Seminars 


Response 


Pretest 


Posttest 


Pretest 


Posttest 


Pretest 


Posttest 


Definitely popular publication 


7 


4 


2.5 


2.9 


2.96 


2.95 


Somewhat popular publication 


4 


5 










I can't make up my mind 


0 


1 










Somewhat scientific Journal 


2 


3 










Definitely scientific Journal 


4 


4 






















• 


B. If it ever came to a choice between working on the practical problems of vocational education 


( problems important to the local schools), or 


contributing to the development of a body of 


scientific knowledge, which would you prefer to do? 


















Means 








Distribution 


North Carolina 


All 


Seminars 


Response 


Pretest 


Post test 


Pretest 


Posttest 


Pretest 


Posttest 


Definitely practical problems 


8 


7 


2.4 


2.1 


2.74 


2.51 


Somewhat practiced problems 


4 


7 










1 can't make up my mind 


0 r 


0 










Somewhat scientific knowledge 


1 


0 










Definitely scientific knowledge 


4 


3 










C. How important to you in a Job is 


the chance to serve people, i.e., 


to help solve their problems? 


- 


* 






Means 








Distribution 


North 


Carolina 


All Seminars 


Response 


Pretest 


Posttest 


Pretest 


Posttest 


Pretest 


Posttest 


Of utmost importance 


11 


9 


1.5 


1.6 


1.60 


1.55 


Very important 


3 


6 










Somewhat important 


3 


2 










Not very important 


0 


0 










Unimportant 


0 


0 


< 
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Research Design 



- Cornell - 



Program 

Vhis seminar was conducted at Cornell University, May 2-6, 1966, with Frederick 
Tom serving as Seminar Director. The number of applicants for the research design 
seminar continued to be greater than could he accommodated. A very large percentage 
of the applicants accepted invitations, with 39 attending, plus five staff from the 
U.S. Office of Education. 



In the three previous years, a similar seminar had been conducted. The good 
features of the preceding programs were incorporated in this seminar, with emphasis 
on experimental design. Two consultants used the major part of the available time 
for a detailed discussion of (l) experimental research and proposal -writing and 
(2) statistical tools and analysis of variance. The major topics, and consultants 
for each, follow: 



The Role of Experimental Research 
in Vocational Education 



Jason Millman 
School of Education 
Cornell University 



Some Perspectives on the 
Research Process 



David Krathwohl, Dean 
School of Education 
Syracuse University 



The Research Process and Its David Krathwohl 

Implication for Proposal Writing ^ 



The Preparation of Research David Krathwohl 

Proposals and the Design of 
Experimental Research 



The Arithmetic of Analysis of 
Variance 



Topics in the Analysis of 
Variance 

Sampling Procedures in Research 

The Analysis of Variance in 
Research 

Current Research Developments in 
The U.S. Office of Education 



Latest Developments in Computer 
Science with Implications for 
the Educational Researcher 



Donald G. MacEachern 
College of Education 
University of Minnesota 

Donald G. MacEachern 



Donald G. MacEachern 
Donald G. MacEachern 



Sidney High 

Specialist in Research 
U.S. Office of Education 

Karl L. Zinn 

Center for Research on Learning 
and Teaching 
University of Michigan 
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„ Four small group sessions studied four selected research proposals in vocational 
education. These sessions were conducted by Eunice Jones, Sherrill McMillen, 

Virginia Thomas and Jack Wilson of the U.S. Office of Education. 

One morning was spent to present experimental studies being conducted in 
Nebraska, Pennsylvania, Minnesota and George Washington University by John Coster, 
Elizabeth Ray, David Pucel and John Dailey, respectively. ' 

The last highlight on the seminar program was a review of research proposals 
submitted by seminar participants. Sidney High served as Chairman of a panel 
including Donald MacEachem, Sara Blackwell, Harold Cushman and D. Bob Gowin. The 
outcome of this activity may be best described by saying that the time available 
was entirely too short to permit answering all the questions of the participants 
and to let them express their reactions to review procedures. 
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Evaluatio n 

Satisfactions . The data presented in Table 24 represent the frequency distribution and median 
for each of the thirty statements of the Satisfactions evaluation form. One-half of these statements 
were presented in the positive form with the others in the negative form. Rather than presenting 
a lengthy discussion of all these data it is suggested that the reader study the distribution of 
responses as well as the median for each of the statements. The following will therefore mention 
only the highlights. 

A high degree of agreement resulted for each positive statement, and a high degree of disagree- 
ment for each negative statement presented to this seminar group. The positive statements resulting 
in a median of 4 or more were l w 7, ^11^ Uu 23, 24 and 29. The negative statements having a median 
response of less than 1 were 6, 8, 26, 28 and 30. 



Knowledge . The data presented in Table 25 gives the distribution of changes in scores from 
pretest to posttest. The first row gives the frequency of changes for scores computed on it ems 
which appeared both on the participants’ pretest and posttest. The second row gives the frequency 
of changes from the participants’ pretest score to the score for all items on the posttest. Also 
shown in this table is the distribution of posttest scores. 

A total of 27 participants increased their scores on the items which appeared on both their 
pretest and posttest forms. There were, however, 9 with no change and 7 with lower scores. The 
distribution of changes between pretest and all item posttest scores was very similar, with l6 again 
showing no change or a loss, while 27 gained. 

The highest scores were in the 80’s, with most participants scoring between 50* and 69*. 



Table 25. Change in Content Knowledge Scores from Pretest to Posttest 

- Cornell Seminar - 









Number of Persons Showing 






Decreased 


No 




Increased Score by: 




Score 


Change 


0 

H 


20% 30* 


40% or more 


Change in scores for items 
on both pretest and posttest 


7 


Q 


l4 


5 6 


2 


Change in scores from both 
pretests to all items on 
the posttest. 


8 


8 


11 


10 3 


3 



Frequency Distribution of Posttest Scores 



Percentage 

score 

Frequency 



01-19* 

0 



20-29* 

0 



30-39* 

1 



4o-49* 

7 



50-59* 

15 



60-69* 

11 



70-79* 80-89* 90-99* 

7 3 0 



Table 24. Median.and Frequency Distribution 
for Degree of Agreement with Statements of Satisfaction 

- Cornell Seminar - 



Frequency Distribution 



Strongly Un- Dis- Strongly 

Agree Agree decided agree Disagree 

Statements Median (5) (4) (3) (2) (1) 



In regard to this conference I feel that: 

1. The purposes of this program were clear to me. 

2. The objectives of this program were not 

realistic 

3. Specific purposes made it easy to work 

efficiently. 

4. The participants, accepted the purposes of 

this program ,. 

5. The objectives of this program were not the 

same as my objectives 

6. I didn't learn anything new . . . 

7- The material presented was valuable to me. . . 

8. I could have learned as much by reading a book 

9. Possible solutions to my problems were 

considered 

10. The information presented was too elementary . 

11. The speakers really knew their subjects. . . . 

12. The discussion leaders were not well prepared. 

13. I was stimulated to think objectively about 

the topics presented 

14. New acquaintances were made which will help 

in future research 

15. We worked together as a group 

16. We did not relate theory to practice ..... 

17. The sessions followed a logical pattern. . . . 

18. The schedule was too fixed 

19. The group discussions were excellent 

20. There was very little time for informal 

conversation .... 

21. I did not have an opportunity to express my 

ideas 

22. I really felt a part of this group ...... 

23. tyy time was well spent 

2 4. The program met my expectations , . 

25. I have no guide for future action. 

26. Too much time vhs devoted to trivial matters . 

27. The information presented was too advanced . . 

28. The content presented was not applicable to 

research in vo-ed 

29. Seminar's of this nature should be offered 

again in future years. ....... 

30. Seminars such as this will contribute little 

to vo-ed research * . . . . . . . . . 



4.0 


22 


22 


0 


0 


0 


1.1 


1 


3 


1 


19 


20 


3.6 


10 


31 


1 


2 


0 


3.8 


16 


27 


0 


1 


0 


1.2 


2 


0 


3 


21 


18 


0.7 


1 


1 


0 


Q 


33 


4.1 


25 


18 


1 


0 


0 


0.8 


1 


1 


0 


l4 


28 


3.5 


6 


31 


T 


0 


0 


1.3 


1 


1 


2 


24 


16 


k.2 


28 


16 


0 


0 


0 


1.4 


0 


1 


5 


24 


14 


3.9 


18 


24 


1 


1 


0 


4.1 


23 


19 


2 


0 


0 


3.8 


15 


27 


2 


0 


0 


1.4 


0 


3 


2 


27 


12 


3.9 


20 


23 


1 


0 


0 


1.3 


1 


2 


6 


17 


18 


3.5 


10 


23 


6 


5 


0 


1.4 


1 


3 


2 


24 


14 


1.1 


0 


0 


1 


24 


19 


3.9 


19 


22 


1 


2 


0 


4.3 


30 


13 


0 


1 


0 


4.0 


21 


19 


3 


0 


1 


1.0 


0 


0 


2 


20 


22 


0.9 


0 


1 


1 


16 


26 


1.6 


0 


5 


6 


23 


10 


0.9 


0 


0 


2 


18 


24 


4.4 


.36 


7 


0 


1 


0 


0.6 


1 


0 


0 


9 


34 



Attitudes. The data presented in Table 2 6 are the pretest and posttest means for this seminar 
and the means for all seminars giving equal weight to each seminar. Figure 5 represents the direction 
and magnitude of change, in hundredths of a point, from pretest to posttest. The reader must consider 
the table and the chart together as well as the relationship between the item means for this seminar 
and item means for all the seminars, (it must also be noted that the direction of change from pretest 
to posttest response for the two negative items, i.e., invalidity and contamination, ha.e been inverted 
so as to be consistent with the 28 positive items. ) 

As was the case in presenting the data derived from the Satisfactions evaluation form, only the 
highlights of these data will be presented below. It is suggested that study of the data presented in 
both the table and chart will be most informative. 

In this seminar only nine research terms received a pretest mean lower than the mean for all 
seminars. As shown by the chart, however, the shift from pretest to posttest was in a positive 
direction. 

The research terms receiving a posttest mean response of 6 or more were: dissemination, applied 
research, research design, experimental research, research findings, evaluation, review of literature, 
randomization, statement of problem, and reliability. The terms receiving less than 5 on the posttest 
were: non-parametric , diffusion research, and nominal scaling, in addition to the two negative terms. 

The direction of change from pretest to posttest was certainly in the positive direction with 
only the term of data processing showing a negative shift of any great magnitude. The terms with 
greatest positive shift were action research, hypotheses, and randomization. 
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Table 26. Change in Attitude* toward Research Terms 



O 

l]( 



* 




- Cornell Seminar 




- - % 




* 








Mean 








Cornell 




All Seminars 


Research Term 




Pretest Posttest 


Pretest 


Posttest 


Variance 




* 5.33 


5.28 


5.272 


5.315 


Non Parametric 


. 


4.82 


4.85 


4.-794 


4.905 


Sampling 




5.80 


5.84 


' 5.748 


5.828 


Dissemination 




6.03 


6.02 


6.014 


5.955 * 


Diffusion Researca 




4.70 


4.' 0> 

b - ' 


4.764 


5.003 


Applied Research 




6.26 


6.20 e . 


• 6.032 


5.965 


Invalidity 




3.32 


2.89 


3.044 . 


3.153 


Funding 




5.69 


5.87 


5.450 


5.785 


Action Research 




5.55 


5.96 


5.674 


5.763 


Research Design 




6.10 


6.09 


5.790 


5.770 


Developmental Research 




5.7? 


5.94 


5.756 


5.766 


Basic Research 




5.85 


5.94 


5.712 


5.703 


Experimented Research 




5.94 


6.14 

i 


5.956 


6.040 


Operational Definition 




5-59 


5.72 


5.480 


5.608 


Nominal Scaling 




4.69 


4.83 


4.726 


4.956 


Research Proposal 




5.73 


5.82 


5.572 


5.620 


at* 

Measurement 




5.82 


5.92 


5.808 


5.751 


Hypotheses 




5.46 


5.99 


5.670 


5.688 


Research Findings 




5.89 


6.10 


5.800 


5-746 


Evaluation 




5.84 


6.06 


5.760 


5.771 


Contamination 


. - 


- 3.02 


2.99 


2.944 


2.958- , 


Inference 




5.09 


5.37 


5.266 


5.216 


Generalization 




5.26 


5,. 50 


5.276 


5.326 


Review of Literature 


* 


5.80 


6.00 


5.620 


5.593 


Data Processing 


. : ; ' 


5.86 


■ i' >' 

5.ii 


5.746 


5.753 ' 


Randomization 


i ! 


5.33 


6.01 


5.354 


$.596 


Theoretical Framework . 


, ,V ■ 


5.79 


5.85 


5.734 


5.505 


Statistician 




5.55 


5.64 


5.462 


5.578 


Statement of Problem 


('■ . 


: 5.98 


6.08 


5.924 


5.851 


Reliability 


•• • *■ J 


5.75 


6.06 


5.742 

•, \ 


5.871 
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Applied vs. Basic Research Orientation . The data in Tables- 27 and 28 give the frequency of 
responses to each of three questions which were given to participants both in pretest and posttest. 
The tables also indicate the mean response per question for this group, as well as the mean response 
for all seminars, giving equal weight to each seminar. . 

. participants in this seminar also indicated on the posttest that their personal preference 
for research was more basic than the preferred orientation of their organisation, which in turn was 
more basic than its actual orientation. The largest shift from pretest to posttest occurred on 
question 3, which indicated a change of preference toward basic research. 

There seems to be no change in the type of publication in which participants would prefer to 
report their research, as indicated in Table 28. There was, however, a slight shift toward practical 
problems in question 2. The chance to serve people, as indicated by question 3, remained very 
important . 



Table 27. Means and Frequency Distribution of Responses vo Lazarsfeld's 
Form of Applied vs. Basic Research Orientation 

% 

- Cornell Seminar 

4 

A. Where would you locate the orientation or prime concern of the organization with which you are 
presently working? 



Response Frequency 



Mean 



Applied 


(l) 


(2) 


(3) 


w 


(5) 


Basic Cornell 


All Seminars 


Pretest 


5 


9 


8 


0 


0 


2.1 


2.16 


Posttest 


4 


7 


10 


l 


0 


2.4 


2.15 


B. Where bn the same continuum would you prefer the orientation of your organization to be located? 






Response Frequency 






Mean 


Applied 


CD 


(2) 


(3) 


(U) 


(5) 


Basic Cornell 


All Seminars 


Pretest 


0 


10 


12 


0 


0 


2.5 


2.66 


Posttest 


3 


6 


12 


l 


0 


2,5 


2.40 


C. Where on this continuum would you like to 


do research? 












Response Frequency 






Mean 


Applied 


(1) 


(2) 


(3) 


(U) 


(5) 


Basic Cornell 


All Seminars 


Pretest 


2 


12 


6 


1 


0 


2.2 


2.54 


Post test 


k 


7 


8 


2 


0 


2.7 ' 


J 

2.61 



4 ' " 
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Table 20, Means and Frequency Distribution of Responses to Storer’s Scale 
of Basic vs. Applied Research Orientation 



- Cornell Seminar 



A. If you had to choose between reporting your research in a popular publication where laymen would 
see it and perhaps use your findings, or reporting it in a scientific Journal, which would you * 
prefer? 



Means 



Response 



Distribution 
Pretest Fosttes'C 



Cornell 



Pretest Posttest 



All Seminars 
Pretest Posttest 



Definitely popular publication 
Somewhat popular publication 
I can’t make up my mind 
Somewhat scientific Journal 
Definitely scientific Journal 



3.3 



3.3 



2.96 



2.95 



B. If it ever came to a choice between working on the practical problems of vocational education 
(problems important to the local schools ) , or contributing to the development of a body of 
scientific knowledge, which would .you prefer to do? 



Means 



Response 



Distribution 
Pretest Posttest 



Cornell All Seminars 

Pretest Posttest Pretest Posttest 



Definitely practical problems 
Somewhat practical problems 
I can’t make up my mind 
Somewhat scientific knowledge 
Definitely scientific knowledge 



6 

4 

0 

7 

4 



4 

7 

1 

9 

0 



3.0 



2,7 



2.74 2.51 



C. How important to you in a Job is the chance to serve people, i.e., to help solve their problems? 

. ■ <•: •• ! *' ' •> '• . , . • Means 



Response 



Distribution Cornell 5 1 All Seminars 

Pretest Posttest Pretest Posttest , Pretest Posttest 



Of utmost importance 
Very important 
Somewhat important ; fj 
Not very important 
Uhimportant 






10 

11 

■i • 1 o 

- 0 
0 



12 

2 

: o 

0 



1.5 



1.4" * 



1.60 



1.55 
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Curriculum Evaluation * 
- Illinois - 



Program 

The Curriculum Evaluation Seminar was held at the University of Illinois, 

May' 16-20, 1966, with William Schi 11 serving as Seminar Director. Interest and 
expectations were high for this seminar as indicated by the large number of appli- 
cants and by the fact that all of those invited, accepted. Shortly before the 
seminar began, three individuals reported that they could not attend. Persons on 
the alternate list were also not able to come. The original purpose of the seminar 
was to explore and discuss variables and paradigms related to the assessment of 
curricular offerings in vocational education. 

The program followed a rigid pattern of presenting major papers, with one to 
three individuals giving responses. Afterwards the participants had an opportunity 
to ask questions or to make comments. Before the seminar, the Seminar Director 
had obtained copies of the major speeches and of most of the responses, and had 
duplicated copies for distribution to the participants at the beginning of the 
seminar. 



One half-day was scheduled for each major presentation with its responses, 
followed by questions and discussion. The topics and speakers were: 

Harry Browdy 
College of Education 
University of Illinois 

Sociology and Curriculum David Street 

Evaluation Department of. Sociology 

University of Chicago 



Problems and Prospects in 
Vocational Education 



1 

Criterion Problems and Donald Leton 

Curriculum Evaluation Bureau of Educational Research 

University of Hawaii 



Psychological Aspect of 
Curriculum Evaluation 



The Countenance of 
Educational Evaluation 



Evaluation Problems of the 
UICSM Curriculum Project 

Administrative Aspects of 
Curriculum Evaluation 



David Ausubel 
College of Education 
University of Illinois 



Robert Stake 
College of Education 
University of Illinois 



John Easley , Jr. 
College of Education 
University of Illinois 



James Lipham 
School of Education 
University of Wisconsin 
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The content of the major presentations served to review curricuiar research and 
developmental projects, and waB scholarly, philosophical, theoretical* and academic. 
Easley's presentation on curriculum development and evaluation was a highlight in 
the program, as it showed much relevance to curriculum development and evaluation in 
vocational and technical education* , 
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Evaluation 



Satisfactions The data presented in Table 29 represent the frequency distribution and median 
for. each of the thirty statements of the Satisfactions evaluation form. One-half of these statements 
were presented in the positive form with the others in the negative form. Rather than presenting a 
lengthy discussion of all these data it is advised that the reader study the distribution of • 
responses as well as the median for each of the statements , The following will therefore mention 
only the highlights. 



In this seminar the median response to positive statements 3, 15, and 19 showed disagreement. 
The other positive statements received a median response on the agreement end of the continuum. 
All negative statements had a median falling in the disagreement categories. The only positive 
statement receiving a median of 4 or better was statement 29. The negative statements 6, 10 and 
30 received a median of 1 or less. - 



Knowledge . The data presented in Table 30 gives the distribution of changes in scores from 
pretest to posttest. The first row gives the frequency of change for scores computed on items which 
appeared on both the participants' pretest and posttest. The second row gives the frequency of 
change from the participants' pretest score to the score for all items on the posttest. This table 
also presents the distribution of posttest scores. 



Here there were 13 participants' With a loss or no change between the pretest score and the 
score on the same items appearing in the posttest. Opposed to this were the 24 persons gaining 10 
or more percentage points. ^The total posttest scores snowed 20 persons improving upon their pretest 
scores, while 10 received lower scores, and 7 remained constant. 



The distribution of total item posttest scores ranged from 20% to 19% with most people 
receiving 50* to 79*. 



Table 30. Change in Content Knowledge Scores from Pretest to Posttest 

- Illinois Seminar - 







Number of Persons Showing 






Decreased No 




Increased Score by; 


■ . 


Score Change * 10% 


20%- 30% 

. .... ... , .. .. .4. 


40% or more 


Change in scores fcr items 
on both pretest and posttest 


6 7 


8 


4 5 


7 


Change in scores from both 
pretests to allitems on 
the posttest . 

■' ' -:-■«! ■ " .. 


10 7 


8 


6 3 


3 

. • 




Frequency Diatribution of Post test Scores 




Percentage t . 

score 01-19* 20-29* 

. 1 ■ ' 5 . ", ' "... 

Frequency 0 1 


30-39* 40-1*9* 


50-59* 


60-69* 70-79* 


80-89* 90-99* 


6 . . 5 . 


10 

. i 


9 6 


0 0 
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Table 29. Median and Frequency Distribution 
for Degree of Agreement with Statements of Satisfaction 

- Illinois Seminar - 



Frequency Distribution 



Statements 




Median 


Strongly 

Agree 

(5) 


Un- 

Agree decided 
(4) (3) 


Dis- 

agree 

(2) 


Strongly 

Disagree 

(1) 


In regard to this conference I feel that: 

* t 

1. The purposes of this program were clear to me. 


3.2 


3 


i9 


6 


8 


1 


2. The objectives of this program were not 

realistic. . . . . . . . T . . 




1.7 


0 


5 


7 


17 


8 


3. Specific purposes made it easy to work 

efficiently. . . „ . . 




2.5 


0 


12 


12 


12 


1 


4. The participants accepted the purposes of 
this program ........... 




3.3 


* 5 


19 


10 


3 


0 


5* The objectives of this program were not the 
same as my objectives. ........... 




1.9 


0 


10 


5 


17 


5 


6. I didn’t learn anything new . . . 


• 


0.8 


1 


0 


0 


11 


25 


7. The material presented was valuable to me. . 


• 


3.9 


15 


21 


1 


0 * 


0 


8. I could have learned as much by reading a book 


1.3 


.1 


0 


2 


21 


13 


9. Possible solutiors to my problems were 

considered . .......... 




3.3 


1 


26 


5 


4 


1 


10. The information presented was too elementary 


• 


0.9 


0 


1 


0 


14 


22 


11. The speakers really knew their subjects. . . 


• 


3.9 


17 


20 


0 


0 


0 


12. The discussion leaders were not well prepared. 


1.6 


0 


1 


10 


l6 


10 


13. I was stimulated to think objectively about 
the topics presented 




3.7 


10 


24 


1 


2 


(0 * 


14. New acquaintances were made which will help 
in future research .... 




3u6 


11 


l6 


4 


4 


2 


15. We worked together as a group 


• 


2.0 


1 


13 


4 


11 


8 


l6. We did not relate theory to practice .... 




1.6 


1 


9 


0 


22 


5 


17* The sessions followed a logical pattern. . . 


• 


3.5 


7 


23 


5 


2 


0 


18. The schedule was too fixed . . . 




1.9 


5 


8 


4 . 


17 


3 


19. The group discussions were excellent .... 




1.9 


. 1 


a 


7 


l6 • 


4 


20. There was very iittle time for informal 

conversation . . . . ... . . . . . . . . . 




1.6 


4 


3 


1 


25 


4 


21. I did not have an opportunity to express my 
ideas. ............. *..«... 


• 


1.7 


0 


7 


3 


24 


3 


22. I really felt a part of this group . . . . . 


• 


3.3 


4 


20 


6 


6 


, ) ! 


23. My time was well spent ........... 


• 


3.7 


14 


16 


4 


3 


o 


24. The program met my expectations . . ... . . 


• 


3.3 


v 7 


15 


4 


8 / 


‘ X’.l 


25. I have no guide for future action. . . . . . 


• 


1.5 


0 


2 


5 


21 


9 


26. Too much time was devoted to trivial matters 


• . 


1.3 


0 


1 


2 


J 22 


12 


27. The information presented was too advanced . 


• 


1.5 


2 


1 


2 


25 


7 


28. The content presented was not applicable to 
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research in vo-ed. » . . . . . . . . . . . . 


• 


1.4 


0 


2 


0 


26 


9 


29. Seminars of this nature should be offered 
again in future years. . . . . . . , . * . . 


• ' 


4.2 


23 


9 


4 


1 


0 


30. Seminars such as this will contribute little 
to vo-ed research. , . > TV-. . . T ... . . 
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Attitudes. The data presented in Table 31 are the pretest and posttest means for this seminar 
and the means for all seminars, giving equal weight to each seminar. Figure 6 represents the 
direction and magnitude of change (in hundredths of a point) from pretest to posttest. The reader 
must consider the table and the chart together as well as the relationship between the item means 
for this seminar and the item means for all the seminars. (It must, also be noted that the direction 
of change from pretest to posttest response for the two negative items, i.e., invalidity and con- 
tamination. have been inverted so as to be consistent with the 28 positive items.) 



As was the case in presenting the data derived from the Satisfactions evaluation form, only the 
highlights of these data will be presented below. It is suggested that study of the data presented 
in both the table and chart will be most informative./ 



This seminar had 13 items which were given pretest responses with means lower than any other 
seminar. It also' had 24 items with pretest means less than the overall means. This might then 
suggest the possibility of another regression toward the mean resulting in many positive changes, 

mi* • _ i h « 4 < 4 * neea on wrtn + Vi T.ro c o c wiimVt nocrof.ivfi qVi -F- h oc* iro qVi t *P+ 



This, hovwer, seems not to be the case since there was as much negative shift as positive shift 
as indicated in the chart. 



The only term receiving a posttest mean of 6 or more was experimental research} whereas the 
terms non-parametric, diffusion research, nominal scaling, and the two negative terms received 
posttest means less than 5. 



The direction of change indicated by the chart seems as much negative as positive with the 
terms non-parametric, funding, and experimental research showing the greatest positive change. 
The terms dissemination, research design and evaluation received the greatest negative change. 
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Table 31. Change In Attitudes toward Research Terms 
- Illinois Seminar *. 





’’ 




Mean 


, - - • 


■\ 


Illinois 


All Seminars 


Research Term 


Pretest 


Posttest 


Pretest 


Posttest 


Variance 


5.25 


5.15 


5.272 


5.315 


Won Parametric 


4.22 


4.78 


4.794 


4.905 


Sampling 


5.5 6 


5.64. 


5.748 


5.828 


Dissemination 


6.27 


5.79 


6.014 


5.955 


Diffusion Research 


4.82 


.4.77 


4.764 


5.003 


Applied Research 


6.00 


5.76 


6.032 


5.965 


Invalidity 


3.33 


3.45 


3.044 f 


3.153 


Funding 


5.19 


• 5.57 


5.450 


5.785 


Ac+ion Research 


5.34 


5.31 

‘ j 


5.674 


5.763 


Research Design 


5.84 ‘ 


5.47 


5.790 


5.770 


Developmental Research 


5.67 


5.59 


5.756 


5.766 


Basic Research 


5.66 


5.72 


5.712 


5.703 


Experimental Research 

. -1 . 


5.81 


6.4i 


5.956 


6.040 


Operational Definition 


5.29 


5.58 


5.480 


5.608* 


Nominal Scaling 


4.60 


4.81 


4.726 


4.956 


Research Proposal 


5.54 


5.50 


5.572 


5.620 


Measurement 


5.62 


5.72 


5.808 


5.751 


Hypotheses 


5.62 


5.63 


5.670 


5.688 


Research Findings 


5.67 


5.62 


5.800 


5.746 


Evaluation 


6.13 


5.68 


5.760 


5.771 


Contamination 


3.34 


3.02 


2.944 


2.958 


Inference 


5.06 


5.08 


5.266 


5.216 


Generalization 


5.53 


5.22 


. 5.276 


5.326 


Review of Literature 


5.87 


5.59 


. - ' 5.620 


5.593 


l % ‘ • ” t _ '*£ 1 - 

Data Processing 


... ; i 5.39 


*5.69 


5.746 


5.753 


Randomization 


5.26 


5.52 


5.354 


5.596 


Theoretical Framework 


5.43 


5.29 


5.734 


5.505 ' ; 


Statistician 


5.26 

v V. 


/C. 5.54 ' ' 


5.462 


5.578 


Statement of Problem 


5.63 


5.67 


5.924 


5.851 


Reliability ' . : : J 


5.73 


5.72 


vi C" 5.742 


5.871 
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Ap plied vs. Basic Research Orientation . The data in Tables 32 and 33 give the frequency of 
responses to each of three questions which were given to participants both in pretest and posttest. 
The tables also indicate the mean response per question for this group as well as the mean response 
for all seminars, giving equal weight to each seminar. 

It is interesting to note that the responses to the questions in Table 32 show a diversion from 
what has been found in the other five seminars in that the personal preference for research is more 
applied than the preferred concern for the participants * organizations. The preferred orientation 
of their organizations was, however, more basic than they Judged it to be at .present. 

Table 33 indicates a slight shift toward popular publications for reporting research. There 
was little change in the type of problem they would prefer to work upon , and again the chance to 
serve people was of great importance’ both in pretest and posttest. 



Table 32. Means and Frequency Distribution of Responses to Lazarsf eld’s 
Form of Applied vs. Basic Research Orientation 

- Illinois Seminar - . 



A. Where would you locate the orientation or prime concern of the organization with which you are 
presently working? 



Response Frequency Mean 



• • • • ' 


Applied 


(1) 


(2) 


(3) 


(4) 


(5) 


Basic 


Illinois 


All Seminars 


Pretest 




2 


9 


5 


2 


0 




2.4 


2.16 


Posttest 




2 


12 


2 


2 


0 




2.1 

r 


2.15 


Where on the 


same continuum would you prefer the orientation of your organization to be located? 




. i 




Response Frequency 








Mean 




Applied 


(1) 


(2) 


(3) 


( 4 ) 


(5) 


Basic 


Illinois 


All Seminars 


Pretest 




1 


IK_ 


5 


6 


0 




2.9 


2.66 


Posttest 




2 


8 

t * ' 


4 


4 


~0 




2.6 


2.40 


Where on this 


continuum would you like to do research? 
















Response Frequency 








Mean 




Applied 


( 1 ) 


(2) 


( 3 ) 


( 4 ) 


(5) 


. Basic 


Illinois 


All Seminars 

, . . j - , 


Pretest 




4 


k 


7 


2 


1 




v 2.6 


2.54 


Posttest i 


* ' .. 


4 


7 


3 


4 


0 




2.4 


2.6l 



Table 33. Means and Frequency Distribution of Responses to Storer's Scale 
of Basic vs. Applied Research Orientation 

- Illinois Seminar - 



A. If you had to choose between reporting your research in a popular publication where laymen would 
see it and perhaps use your findings, or reporting it .in a scientific Journal, which would you 
prefer? 



Means 

Distribution Illinois . All Seminars 

Response Pretest Posttest Pretest Posttest Pretest Posttest 



Definitely popular publication 2 
Somewhat popular publication 5 
I can't make up my mind 1 
Somewhat scientific Journal 5 
Definitely scientific Journal 5 



3 

6 

1 

5 

3 



3.3 2.9 



2.96 2.95 



B. If it ever came to a choice between working on the practical problems of vocational education 
(problems important to the local school), or contributing to the development of a body of 
scientific knowledge, which would you prefer to do? 



Means 

Distribution Illinois All Seminars 

Response Pretest Posttest Pretest Posttest Pretest Posttest 



Definitely practical problems 8 
Somewhat practical problems 3 
I can't make up my mind 1 
Somewhat scientific knowledge 3 
Definitely scientific knowledge 4 



6 2.6 2.7 2.74 2.51 

5 
1 

3 • 

4 



C. Hoy important to you in a Job is the chance to serve people, i.e., to help solve their problems? 



Response 



Of utmost importance 
Very important 
Somewhat important 
Not very important 
Uhimportant 



Distribution 
Pretest Posttest 



8 7 

8 7 9 

3 2 

0 1 

0 0 



Means 

Illinois All Seminars 

Pretest Posttest Pretest Posttest 



\ 1.7 1.6 1.60 1.55 




CONCLUSIONS AND IMPLICATIONS 



The analysis of evaluation data and general observations of the project are 
the basis to support the conclusions and implications which follow. The first part 
of this section relates to the evaluation of the seminars. The second part deals 
with general observations. 



Evaluation 



The evaluation of the seminars gives general indication of successful accom- 
plishment of the purposes set forth for the seminars. Each type of evaluation 
exhibited many similarities as well as differences in seminars. The following 
presents conclusions and implications which seem to be indicated by the data 
obtained from the evaluation instruments. 

Knowledge . In general it may be concluded that there was some gain in knowledge 
as indicated by a multiple choice test. Only in the North Carolina and Cornell 
seminars were there much larger numbers of persons showing increases between pre- 
test and all item posttest scores. The Georgia and Illinois seminars showed a 
slight difference in favor of those increasing, whereas the Colorado seminar was 
lacking greatly in increments. There was, of course, a different result when only 
the ten items given to each participant on the pretest were used for posttest scores . 
Here, there were greater numbers of persons increasing their scores for all seminars 
but the one at Colorado. 

There are several factors which may have contributed to the generally disap- 
pointing results in the area of change in knowledge. The first is the difficulty 
of constructing a reliable and valid test from some of the papers to be delivered 
at the seminars. Not only was it impossible to obtain a paper beforehand from a 
few of the consultants, but those that were obtained were not available far enough 
in advance to allow adequate time for optimum test construction. This leads to two 
limitations: (l) not all topics of the seminar were included in the test and (2) 
the actual emphasis given to parts of a topic by the consultant could not be 
identified from the paper itself. These limitations were especially true of the 
Ohio, North Carolina and Illinois seminars. 

Another possible source of limitation may have been the attitude of the 
participants toward being tested on the material presented in the seminar. Most 
participants , since they were professionals , had been removed from the taking of 
tests for several years and may have been less favorably disposed to try hard at 
such an. endeavor. 

A third possibility would be that there had been little change in knowledge, 
in the positive direction, during the week of the seminars. From subjective obser- 
vations, and from comments by the participants, this does not seem to be the case. 
Furthermore, the responses to relevant statements on the Satisfactions evaluation 
form also indicate that this was not true of the seminars. 

These factors indicate to the investigators that either the change in knowledge 
aspect of evaluation be discontinued for future seminars or else reliable and valid 
tests be constructed based upon all topics to be presented and emphasizing those 
parts deemed important by the consultants. The difficulty of attaining this last 
alternative suggests that the first (no evaluation of change in knowledge) be 
accepted. 
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S atisfactions . The data provided by this form of evaluation indicate general 

satisfaction although certain statements did point out specific differences between 
seminars . 



The statement that "Seminars of this nature should be offered again in future 
^ responded to with the greatest degree of agreement - a median response 
°V; f ? f ? r 011 semin&r Participants. The statements "X didn’t learn anything new," 
and Seminars such as this will contribute little to vo-ed research" were disagreed 
with heartily (median of 0.8). ^ 



, « '? is aspect of the evaluation procedures lends great support to the conclusion 
that the seminars met the purpose set forth. It also indicates some directions for 
change in the methods of presenting future seminars . 



Attitudes. The greatest change in magnitude of responses from pretest to 
posttest means was .6l on a 7 point scale. There were, however, different patterns 
in the direction of change between seminars. Though no hypotheses were stated the 
direction and magnitude of attitudinal change seemed directly related to the content 
and methods of presentation in the various seminars. 






As mentioned above these changes might partially be attributed to regression 
toward the mean for two of the five seminars. However, the other three seminars 
(which. were given both pretests and posttests) show patterns which do not lead to 
j° n suc h a statistical artifact. The pretest scores indicated that the 
participants brought to the seminar positive attitudes toward research. This 
increased the difficulty of creating change in a positive direction. 



It is the Judgment of the investigators that any greater shifts in attitude 
toward research will require more than 5 days ’ time or else a more directive type 
° f 1 ?^ Vlty * Also » the identification of attitudinal change is a function of the 
validity of the attitude measurement device used. In the Judgement of the investi- 
gators further effort should he expended both in the construction of better 
measuring devices and in the attempt of identifying ways and means of changing 

flT.T.I Tll/IAO ° ® 



^gplied vs. Basic Research Orientation . Two conclusions regarding orientation 
toward basic and applied research seem Justified by the data obtained. The first 
is that little change was noted between pretest and postttest administration of 

ei l er H P °? scales * orientation reflects a complex value system on the 

part of individuals and changes of any great extent are not expected in such a short 
period of time. 



The second conclusion indicated by these data is the very applied orientation 
o participants. On the posttest of the Lazarsfeld instrument the applied end of 
the continuum was checked for the first three questions by 99, 78 and 82 participants 
respectively. Opposed to this only 13, 12 and 27 check the basic end of the con- 
tinuum for questions I through 3, respectively. 



The Storers instrument on posttest gave a somewhat different picture in that 
90 persons preferred to work on practical problems as opposed to 49 who indicated 
preference for contributing to a body of scientific knowledge. This ratio of 
almost 2 to 1 was not observed in question 1 which asked in what type of publica- 
tion they would prefer to publish their research reports. The distribution here 
was roughly even with 73 prefering popular publications and 67 scientific Journals. 
As^could be ejected, great importance was given to the chance to serve people as 
indicated by question 3 where 131 persons checking the utmost or very important 

categories with only 15 responding in the remaining 3 categories. 



V 
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A question is raised here by the investigators , as to how we may best serve 
people. In vocational education, will we best serve people by continuance of this 
strong orientation toward applied research? It seems that a strong case could be 
made to support a more basic approach which in the long run may provide greater 
service to more people. This question is worthy of further consideration and study. 




Application of outcome of the seminars . From a list of ten ways participants 
might use the outcome of the seminars, they were asked to check the two most 
important. Their responses are given in rank order for the seminars in Table 3^. 
The objectives of the seminars, the interest and orientation of the individuals 
influenced the rankings — as may be seen in the variations from one seminar to 
another. The administration of research programs ranked first for the Ohio seminar 
and sixth for all seminars. Somewhat interesting to note is that the increased 
ability to advise others in research planning rank first for all seminars. The 
preparation of research proposals and use in present research projects ranked 
second and third for the six seminars. The preparation of curriculum materials 
ranked third for the Georgia and Illinois seminars, but was eighth for the overall 
rank. The improvement of teaching or supervision ranked fifth for all seminars. 
This was higher than might be anticipated from seminars on research in vocational 
education. It may be stated, with the few exceptions pointed out above, that the 
rankings of the participants* plans to apply results of the seminar are in line 
with the objectives and content of each seminar. 
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Table 34. Ranking of the Ways Participants Plan to Apply 

Outcome of Seminars 



Average 






Seminar Ranking 






Type of Application 


Rank 


Ohio 


Georgia 


Colorado 


N . Carolina Cornell 


Illinois 


Increased ability to 
advise others in 
research planning 


1 


2 


1 


1.5* 


4 


2 


3* 


Preparation of a 
research project 


2 


3.5* 


2 


1.5* 


3 


1 


7 


Use in present 

research projects 


3 


* 6 


5* 


3 


1 


4 


1 


Increased knowledge of 
research in general 


4 


3.5* 


5* 


5.5* 


7* 


3 


5.5* 


Improvement of teaching 
or supervision 


5 


5 


5* 


4 


5 


5.5* 


5.5* 


Administration of 
research programs 


6 


1 


7.5* 


5.5* 


7* 


5.5* 


8 


Planning of vocational 
training programs 


7 


7.5* 


7.5* 


7 


2 


7 


3* 


Preparation of 

Curriculum materials 


8 


9.5* 


3 


9,5. 


7* 


9* 


3* 


Planning of vocational 
education facilities 


9 


7.5* 


9.5* 


9.5* 


9 


9* 


9.5* 


Writing an article or 
other publication on 
this topic 


10 


9.5* 


9.5* 


8 


10 


9* 


9.5* 



* Indicates tie rankings. 
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General Observations 

In the opinion of the investigators the vocational education research seminars 
should he continued. The development of seminar programs around problem areas has 
been feasible and it should be continued. Vocational educators request that seminars 
be planned on topic or problem areas as well as research methodology and statistics. 
Thus, providing a choice to meet the needs of the individuals. 

In the planning and development of a research seminar program, the host university 
planning committee served a very useful and worthwhile purpose. A committee should 
continue to advise and assist the seminar director. The overall project director 
should take an active role in advising and assisting the seminar director in 
structuring and organizing the program. A variety of methods and techniques should 
be structured into the program to the end that each member of the group becomes 
actively involved in a variety of learning activities. 

The method used in the selection of participants worked reasonably well. The 
responsibility should be coordinated by one person. The following would improve 
and make for a more effective procedure. An announcement giving the content of 
the seminars should be developed so that applicants would know the specific content 
in each. Selection criteria should be developed and approved by a selection com- 
mittee. The announcements, application form and selection criteria should be given 
wide distribution to all personnel in the various branches of vocational education. 

The selection of participants is becoming increasingly important. While the 
seminars do generate some interest in research there were a number of vocational 
educators who had not been involved in research projects nor were they engaged in 
a project at the time. Adequate data on an application form could go a long way in 
making a decision to accept or reject applicants. The reasons for attending a 
seminar along with other data would give a basis for selecting participants . A 
homogeneous group with similar research abilities and interest makes for an effective 
working group. The success of a seminar is dependent upon the above as much as or 
more than any other factor. 

In planning of a program that includes consultants from various disciplines, 
it is most important to orient the consultant to the application and use of the 
presentation. Then, the consultant or seminar director should follow up with the 
group so that they relate and make application of the content to research. More 
than a lecture is required to develop abilities and attitudes needed in research. 

Two seminars in which the lecture method prevailed were considered to be less satis- 
factory by the participants than those in which a variety of methods, techniques and 
procedures were used to involve the participants. A week of mostly lectures is to 
be avoided. 

In view of the statements above it must be said that the consultants in the 
academic disciplines, education and vocational education, with a few exceptions 
gave excellent presentations. A mix when appropriate from all three can make a 
desirable contribution. Each has a place and can make a contribution. 

Two host university seminar directors did an exceedingly fine job in obtaining 
copies of the major presentations from the consultants and duplicating copies for 
the participants prior to the seminar. In one seminar the papers were distributed 
at the beginning of the seminar. In the other, papers were passed out after the 
speakers gave their presentation. The majority of the participants in one seminar, 
where speeches were given out in advance of the presentation, preferred this to 
distribution afterwards. 
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Summary of Project 0E-6-85-027 




Title: The Further Development of Research Competencies of Personnel in 

Vocational Education Research and Development 

Investigator: C. W. Hill, Professor, Agricultural Education 

Institution: Cornell University 

Ithaca, New York 

Duration: September 1, 1965 to August 31, 1966 

Purposes 

The major objective of this project was to develop further the research 
knowledge, competencies and interests of individuals engaged in, or soon to become 
engaged in, research in the field of vocational education. Six problem-centered 
areas were identified for a one-week research seminar in each of six universities. 



Procedure: 



The Research Committee of the American Vocational Association, in cooperation 
with vocational education staff in the U.S. Office of Education and selected uni- 
versities initiated and conducted a series of vocational education research seminars 
beginning in 1963. The first was conducted on research design at Purdue in 1963. 

The following year two were held, one at University of Illinois and the other at 
Pennsylvania State University, In 1965 the U.S. Office of Education approved a 
project submitted by the University of Illinois and financed under P.L. 88-210 
Section 4(c) for four research seminars. 

The Research Committee of AVA asked Cornell University to develop a proposal 

and seek approval of a project for coordinating and funding the vocational education 
research seminars in 19 66. 

In a meeting with representatives of the U.S. Office of Education, AVA Research 
Committee, and selected universities, it was decided to conduct six seminars and 
to concentrate upon six problem areas. 

The appropriate administrative staff in the universities were contacted to: 

(1) select and designate a seminar director, ( 2 ) select the date for the seminar, 

(3) decide on the number of participants that could be accommodated, and (4) decide 
upon the objectives and content for the seminar. The seminar directors, with the 
advice and assistance of a planning committee, worked out the plans for a five-day 
seminar in each of the universities. 

• 

An announcement of the seminars was placed in the September 1965 issue of the 
American Vocational Jbursnl. Readers were informed that further InfOnaatlon and 
an application form could be obtained from the Project Director. An announcement 
of the seminars and application forms were sent to state directors of vocational 
education, state directors of Research Coordinating Units, former participants in 
vocational education research seminars and vocational administrators^ universities. 

■ V ' 

A selection committee, composed of the AVA Research Committee and U.S. Office 
of Education staff, reviewed data submitted on application foxms and selected 



w 



***** 









'1; . ■ ’ . *l» 

■ ^ ' ■■■■■ ■ ‘ . ■ ■■ %; , ... ■ • 

4 ,ni : k ti***** 



* 68 

participants for each of the seminars. The lists of participants were sent to each 
of the seminar directors in the host universities, and they extended the invitations 
to attend. 

In the development of the programs, the seminar directors, with some assistance 
of a planning or advisory committee,' sought and obtained the services of consultants, 
nationally known and recognized, who could make an outstanding and significant con- 
tribution to the seminar program. The consultants came from such academic disciplines 
as ^sociology, psychology, economics, research, statistics, philosophy, education, and 
from other areas such as governmental services at state and national levels, voca- 
tional education and public schools. Many of the consultants prepared in advance a 
copy of their presentation, so that the seminar directors were able to duplicate and 
distribute copies to participants .at the seminar. 

The evaluation of the seminars was a part of the approved project. Areas 
selected for evaluation were participants* satisfaction with the seminars, changes 
in knowledge, research attitude, and basic vs. applied research orientation. 
Instruments were adapted or developed to assess changes in each of the areas. A 
Likert-type instrument was used for measuring participant satisfactions. Multiple- 
choice questions were developed to ascertain change in knowledge of subject matter. 

A semantic differential was developed to assess changes in attitudes toward thirty 
research terms. Two types of scales were used to assess basic vs. applied research 
orientation. A pretest was given at «he beginning of the seminars and a posttest at 
the end. 



Results: 



A series of six one-week vocational education research seminars were planned 
and conducted as listed below. 



Seminar 


Date 


Host University and 
Seminar Director 


Number of 
Participants 


Development and Coordination 
of Research by State Research 
Coordinating Units 


January 30- 
February 4, 
1966 


Ohio State University 
Robert Taylor 


56 


Curriculum Development 


February 7-11 
1966 


, University of Georgia 
Herschel Lester, Jr. 


4l 


Tests and Measurement 


March 27- 
April 1, 1966 


Colorado State 
University 
Douglas Sjogren 


44 


Occupational Mobility 
and Migration 


April 17-22, 
1966 


North Carolina State 
University 
Harry Beard 


34 


Research Design 


May 1-6, 
1966 


Cornell University 
Frederick K.T. Tom 


44 


Curriculum Evaluation 


May 16-20, 
1966 


University of Illinois 
William Schill 


39 


In addition to 244 participants from the 


states , 14 staff from the 


i ! • 

U.S. Office 



of Education also attended, making a total of 258 participants for of the seminars. 

These seminars were conducted primarily to further the development of the 
research competencies in vocational educators. The analysis of evaluation data 



and observations by project staff support the conclusions and implications which 
follow • 



The evaluation of the seminars gives general indication that the objectives 
were successfully achieved for the seminars. Each type of evaluation exhibited 
many similarities as. well as differences among the seminars. 

* n Carolina and Cornell seminars there were larger numbers of persons 

showing increases in scores between pretest and on all posttest items. The Georgia 
and Illinois seminars showed a slight gain, whereas, the Colorado seminar was lack- 
xng in increments. There was a different result when only the ten items given to 
each participant on the pretest were used for the posttest scores. Here, there were 
greater numbers of persons increasing their score for all seminars except the one 
® The construction of valid tests, the emphasis placed np^n topics by 

the consultants and the sophistication of the individuals were factors influencing 
the results in achievement. 

The data on participant satisfactions with the seminars lend great support to 
the conclusion that the seminars met the purpose set forth. The Support of this 
varied from seminar to seminar and the degree of agreement between items. Overall 
the purposes, content, methods, techniques and procedures were acceptable. However 
the findings indicate that a sizable number of participants in two seminars were not 
well satisfied with the objectives or the lack of objectives and the structure and 
functions in the operation of the seminars. 

pie data on changes in research attitude derived from the semantic differential 
resulted in different patterns in the directions of change between seminars. In 
one seminar the direction of change was positive. In others the change in attitude 
was positive and negative; while in one seminar the pretest mean was very high 
indicating that the participants brought to the seminar a high positive attitude 
toward research. Then on the posttest the mean for the group declined. This might 
be attributed to regression toward the mean, characteristic of test-retest results 
on the semantic differential. The pretest scores indicated that the participants 
brought to the seminars a positive attitude toward research. This increased the 
difficulty of creating change in a positive direction. It should be recognized 
that to make any significant changes in attitudes, more than five days are required. 

pie data, on the orientation of participants toward basic and applied research 
indicated very little change between pretest and posttest administration of the two 
scales. This orientation reflects a complex value system on the part of individuals 
and changes of any great extent are not expected in such a short period. 

As indicated by a Lazars f eld type of instrument the participants were highly 
oriented to the applied research end of the continuum in contrast to basic research. 
An approximate ratio of two to one preferred to work with practical problems as 
opposed to contributing to scientific knowledge. 

The findings in the evaluation of the seminars and observations strongly 
support the continuation of research seminars for vocational educators. Seminar 
programs should be based upon topics or problem areas in vocational education as 

well as research methodology and statistics. Thus, providing a choice to meet the 
needs of individuals. 

P 1 ® selection of participants is becoming an increasingly important problem. 
Individuals involved or likely to be involved in research should be invited to 

seminars. A homogeneous group with similar research abilities and interests makes 
for an effective seminar. 



***** 
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Appendix 



A. Program and List of Participants for 
Seminars Held at: 



Ohio State University 



University of Georgia 



Colorado State University 



North Carolina State University 



Cornell University 



University of Illinois 
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National Vocational Education Research Seminar 

on 

DEVELOPMENT AND COORDINATION OF RESEARCH 
Bi STATE RESEARCH COORDINATING UNITS 

Ohio State University 

January 31-February 4, 1966 



Robert E. Taylor, Seminar Director 




PROGRAM 



Purpose of the Seminar 

To assist research coordinating unit personnel and other educational leaders in 
fulfilling their roles in planning and conducting comprehensive state programs of 
research tad development in vocational education. 



Objectives 

1. To develop a concept of research and development for state programs of vocational 
education and the conditions essential to its implementation and success. 

2. To develop an understanding of a functional organizational structure for 
establishing and administering a program of research and development in voca- 
tional education. 

3. To identify key individual and organizational roles and clarify their 
relationships to research and development activities. 

4. To develop an understanding of the dynamics of planned change through research 
and its application to vocational education. 

5. To stimulate empathy and support for research and development activities in 
vocational education. 

6. To establish lines of communication for coordinating state, regional and 
national research and development activities in vocational education. 

7. To provide a setting where individuals can share ideas and seek solutions to 
common operational problems in vocational education research and development 

activities. J 
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Monday , January 31 



Chairman: V. E. Christensen, Consultant, The Center for 

Vocational and- Technical Education, The Ohio 
, State University 



8:30 


Registration 




9:15 


Introduction of participants 




9:30 


Welcome to fhe Ohio State 
University 


Alfred B. Garrett 

Vice President for Research 

The Ohio State University 


9:^5 


Overview of the Seminar 


Robert E. Taylor, Director 
The Center for Vocational 
and Technical Education 
The Ohio State University 


10:30 


The Research and Development 
Concept - What Is There to be 
Done and Who Can Do It Best? 


Ray Jongeward 
Director of Research 
Office of Superintendent of Public 
Instruction 
Olympia, Washington 


11:15 


Reactors 


John Bean 

U.S. Office of Education 






Byrl Shoemaker 

Director, Vocational Education 
State of Ohio 


11: Uo 


Discussion period 




1:30 


Implications of Diffusion 
Research to Implemented 
Change in Education 


H. F. Lionberger, Chairman 
Department of Rural Sociology 
University of Missouri 


2:15 


Discussants : 


Alan Robertson, Chief 
, Bureau of Research and Evaluation 
State Education Department New York 


3:15 


Questions and Reactions by 
Participants to Presentations 


H. F. Lionberger 



Tuesday , February 1 



8:30 Improving Research in 

Vocational Education 

Reactor 

10:15 Structuring the Situation for 

Research 



Alan Knox, Professor of Education 
Teachers College 

Columbia University, New York City 

David L. Clark 
Professor of Education 
The Ohio State University 

Loyal Joos 

Director of Systematic Studies 
Oakland Schools 
Pontiac, Michigan 



10:15 con’t.- Reactor 



1:30 Discussion period with panel 

3:15 Small group discussion - 

Establishing Role and Policy 
for Research Coordinating Units 

6:30 Banquet 

Introduction of Guests 

Entertainment 

Address 



Gordon Swanson 

Professor and Coordinator Interna- 
tional Programs 
College of Education 
University of Minnesota 

Alan Knox 
Loyal Joos 
Gordon I. Swanson 
David L. Clark 

Discussion Leaders; 

Ray Jongeward, Washington 
Alan Robertson* New York 
Robert Worthington, New Jersey 

Toastmaster: 

William B. Logan 

Director of Distributive Education 
The Ohio State University 



John Furbay, Guest Lecturer 
Courtesy of General Motors 
Detroit, Michigan 



Wednesday , February 2 

Chairman: Ralph E. Bender, Chairman Department of 

Agricultural Education, The Ohio State University 



8:30 Panel - Major issues in the 
operation and management of 
research coordinating units 

Getting Established 
Structure and Organization 
Staff and Staffing 
Relationships 
Daily Operation 
Moderator 

10:15 Round table discussion groups 

11:00 Discussion period - Panel forum 



Howard Nelson 
University of Minnesota 

Ken Shibita 
University of Nebraska 

Carl Lamar 

University of Kentucky 

Douglas Sjogren 
Colorado State University 

Trevor Howe 

Iowa State Department of Education 
James Wall 

Mississippi State Department of 
Education 



1:30 What Constitutes a Researchable 
Problem and a Respectable 
Proposal 

2:30 Discussion period 

3:15, Round table discussion on the 

problems and issues arising in the 
Research Coordination Units and 
State Vocational research offices 
as to establishing research pri- 
orities and designing proposals 

4:00 Discussion period 

7:00 Evening Session - in University 

School 



Egon Guba 

Professor of Education 
The Ohio State University 

Egon Guba 



Egon Guba 



Putting Data Processing to Work Loyal Joos 
in Vocational Education 



Thursday , February 3 



8:30 



10:15 



11:00 

1:30 

1:45 



3:15 



Chairman: Willis E. Ray, Professor of Education 

Industrial Arts Education, The Ohio State University 



Putting PERT to Work in Research 
Units 

Workshop Sessions 



Discussion period with workshop 
sessions leaders 

Questions period 

Involvement of State Research and 
Development Units in Regional and 
National Projects 

(l) Reports on Current Research 
and Development Activities 

Panel 



(2) The National Program 

Evaluation as called for in 
the Vocational Education 
Act of 1963 

Small group simulation sessions 



Desmond Cook, Professor of Education 
The Ohio State University 

Leaders : 

Desmond Cook, Ohio 
Michael Munger, Nebraska 
John Coster, Nebraska 



Boyd Applegarth, California 
Dick Rice, Ohio 

Harry G. Beard, North Carolina 

Bernard Michael 
Program Evaluation Officer 
Division of Vocational-Technical 
Education 

U.S. Office of Education 

Leaders : 

Boyd Applegarth 
Dick Rice 
Harry G. Beard 
* Bernard Michael 
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4:00 Small groups report back to 
* session 

Discussion period with panel members 



Friday * February 4 

Chairman: Robert M. Reese, Professor of Education, 

Trade and Industrial Education, The Ohio State University 



8:30 Question period 

8:45 Dissemination of Research and 

Development Information 

10:30 Post-Seminar evaluation 
11:15 What Lies Ahead 

12:00 Announcements and close 



Harold Haswell, Director 
Educational Reserve Information 
Center (ERIC) 

U.S. Office of Education 



David Butvhenll, Director 

Division of Adult and Vocational Research 

U.S. Office of Education 



Seminar Planning Committee 



Seminar Director and Director, The Center for Research and 
Leadership Development in Vocational and Technical Education, 
The Ohio State University 

Chairman, Department of Agricultural Education 
The Ohio State University 

Consultant, Research Design' 

The Ohio State University 

Professor, Home Economics Education 
The Ohio State University 

Director of Distributive Education and Professor of Education 
The Ohio State University 

Robert M. Reese Director, Trade and Industrial Arts Education 

The Ohio State University 

Edward R. Towers Associate Professor, Industrial Arts Education 

The Ohio State University 

Inez R. Wells Professor, Business Education 

The Ohio State University 



Robert E. Taylor 

Ralph E. Bender 

<* 

V. E. Christensen 
Marie M. Dirks 
William B. Logan 



Participants 



James C. Atherton 
Joseph K. Bailey 
Phillip Baird 
Edward T. Brown 
Clarence E, Bundy 
V. E. Burgener 
Fairchild Carter 
B. E. Childers 
Evan Confrey 
John Cummings 
Edward F. Davey 
Richard A. Dowd 
Kenneth M. Eaddy 
Everett D. Edington 
Frederic Finsterbach 
Don Frazier 
Melvin H. Garner 
Wayne Grames 
Peter T. Harkness 
Chester A. Hausken 
Rufus W. Hogard 
Trevor G. Howe 
Edward B. Hudgens 
Norman Hyatt 
Minnard H. Jones 
Ronald E. Kaiser 
Charles L. Langdon 
Carl F. Lamar 
Arthur M, Lee 
Kenneth M. Loudermilk 



Austin G. Loveless 

William McNiece 

Philip Masley 

George McCutchan 

Fred Miner 

M(yra Mbsier 

Elwyn H. Nagel 

Howard E, Nelson 

Merrill Redemer 

Fernando Roca de Leon 

John Rolloff 

T. A, Ryan 

Gene Schrader 

Dora R. Sheldon 

A. G. Sheperd, Jr. 

Kenneth E. Shfbata 

Douglas Sjogren 

Bill Stevenson 

Alvin I. Thomas 

Cecil 0. Towers 

George A. Wagner 

James E. Wall 

R. W. Whinfield 

Robert G. Whittemore, Jr. 

Kenneth M. Wold 



John E. Bean 
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National Vocational Education Research Seminar 

on 

CURRICULUM DEVELOPMENT 
University of Georgia 
February 7-11, 1966 
Herschel Lester, Seminar Director 






* 

PROGRAM 



Specific Objectives 

1. To ascertain specific variables which should be controlled in designing 
vocational-technical education curriculum studies. 

2. To outline and criticize curriculum development studies which have employed 
various research methods. 

3. To establish need for participants to engage in curriculum research that would 
• serve as a catalytic agent for designing of future studies. 

4. To review tentative curriculum-development proposals. 



Monday . February 7 



8:00 

8:30 



9:00 



10:30 



1:30 



3:15 



7:00 



Registration 

Introduction and Plans for the 
Seminar 

Curriculum Studies - A Challenge Duane Nielsen 

an Opportunity Educational Resources Development 

Branch 

U.S. Office of Education 



AVA Research Committee and Charles W. Hill, Project Director 

Seminar Evaluation Cornell University 

Presiding: Bill Cheshire, University of Georgia 
Head, Distributive Education 



Labor Market Analysis and 
Projections 



The Interpretation of Economic 
Data 



Evening Activities 
Reception 



Robert E. Mari told 
Office of Manpower, Automation and 
Training 

U.S. Department of Labor 
Norman J. Wood 

Professor and Head Department of Economic 
University of Georgia 
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Tuesday , February 8 



Preaiding: Aleene Cross, Head, Home Economics Education, 
University of Georgia 



8:30 Social Stratification 



10:15 Relationships of Community 

Environment to the Vocational 
Education Curriculum 

1:30 Educational Psychology and 
the Curriculum 



3:15 Currant Research Developments in 
the U.S. Office of Education 

Questions 

7:00 Departmental Activities 



Raymond Payne 
Professor of Sociology 
University of Georgia 

8elz C. Mayo, Director 
Center for Research, Development and 
Training in Occupational Education 
North Carolina State University 

Joe Bledsoe 

Professor of Education 
University of Georgia 

Duane Nielsen 



Wednesday . February 9 

Presiding: G. L. O'Kelley, Jr. , Professor of Agricultural Education 

University of Georgia 



8:30 Statistical Models in Curriculum 
Development Studies 

10:15 The New Mathematics: A Pattern 

for Curriculum Reform 

1:30 Curriculum Development and 

Evaluation in English 

3:00 The Anthropology Curriculum 

Project as a Model for Curriculum 
Development: Practical Problems 

7 : 00 Banquet 

Introductions 

Institutional Research Programs 



Harry E. Anderson, Associate Director, 
Center for Educational Stimulation 
University of Georgia 

Joseph R. Hooten, Jr. 

Professor of Mathematics Education 
University of Georgia 

Mary J. Tingle 

Associate Professor of Education 
University of Georgia 

Marion Rice 

Assistant Professor of Education 
University of Georgia 



J. A. Williams, Dean 
College of Education 
University of Georgia 
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Thursday . February 10 

Presiding: C. C. Calhoun, Head, Eusiness Education 

University of Georgia 

8:30 Panel - Review of Current Curriculum Studies 

1. Development and Evaluation of an Experimental Curriculum for the New 
Quincy, Massachusetts Vocational-Technical School - Edward J. Morrison, 
Director of Vocational Research, American Institutes of Research 

2. An Experimental Evaluation of Approaches to Preparing High School 
Students for Agriculture Occupations Other than Farming - John Coster, 
Director, Agricultural Education Research, University of Nebraska 

3. Occupational Requirements in Office Occupations for School Leavers - 
Fred Cook, Chairman, Business Education, Wayne State University 

4. Evaluation of Secondary School Programs to Prepare Students for Wage 
Earning in Occupations Related to Home Economics - Helen Y. Nelson, 
Associate Professor, Home Economics. Education, Cornell University 

1:30 General Proposal Writings - Warren Findley, Director, Center for 

Educational Stimulation, University of Georgia 

3:15 Panel - Formal Review of Prepared Curriculum Proposals 

James B. Kenney, Assistant Professor of Education, University of Georgia 

Kathryn Blake, Associate Director, Center for Educational Stimulation, 

University of Georgia 

Karl King, Assistant Professor of Family Development, 

University of Georgia 

Charles Johnson, Associate Director, Center for Educational Stimulation, 

University of Georgia 

Leonard Pikaart, Assistant Professor of Mathematics Education, 

University of Georgia 



Friday . February 11 

Presiding: Karl Doss, Head, Trade and Industrial Education : 

University of Georgia 

8: 30 Panel — Fo rm al Review of Participants * Curriculum Proposals 
11:30 Seminar Evaluation 
12 : 00 Luncheon 

Completion of Seminar — H. T. Lester, Jr., Vocational Research, Seminar 

Director 




General Planning Committee 



C. C. Calhoun 
Donald L. Crawford 
Ralph H. Tolbert 
George L. 0* Kelley 
H. T. Lester, Jr., Chairman 



Karl Doss 
A. B. Racster 
Aleene Cross 
Fannie L. Boyd 
. Seminar Director 



Program and Arrangements Committee 

Aleene Cross H. T. Lester, Jr. 

George L. 0* Kelley, Jr. 



Participants 



Hazel Anthony 
Joseph Arnold 
R. A. Baker 
Carl R. Bartel 
Julia M. Boleratz 
June Cozine 
Lucy C. Crawford 
Gordon F. Culver 
Jerry S. Dobrovolny 
Alfred S. Drew 
Jack S. Duncan 
George Ekstrom 
Kenneth A. Ertel 
Edward T. Ferguson, Jr. 
George Fuka 
Anna M. Gorman 
Gilbert S. Guiler 
Wayns House 
Maude A. Hudson 
Thomas A. Tackson 



Mary K. Klaurens 
Eleanore L. Kohlmann 
Frank J. Konecny 
Roland J. Krogstad 
William B. Logan 
Gene M. Love 
Joseph F. Luetkemeyer 
Donald G. Lux 
Alan W. Metcalf 
John L. O' Brian 
David Joseph Pucel 
Henry James Rokusek 
Harlan d E. Samson 
Jacob Stem 
Jerry Streichler 
Philip R. Teske 
Frederick K. T. Tom 
Gail Trapnell 
J. R. Warmbrod 
Earl S. Webb 



Sidney C. High 
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National Vocational Education Research Seminar 

on 

TESTS AND MEASUREMENTS 
Colorado State University- 
March 28-April 1, 1966 
Douglas Sjogren, Seminar Director 



PROGRAM 



Reception 



Sunday , March 27 

Mrs. Juanita Roberts, Hostess 

/ 



Monday , March 28 



Registration 

Orientation Douglas Sjogren 

Charles W. Hill 



Presiding Herbert Benson 

Tests and Measurements in Research J. Stanley- Ahmann 

Workshop Session Number One Charles 0. Neidt 

C. Dean Miller 

Workshop Assistants: David Sheppard 

Charles Stewart 
James Shear d 
Bradley Huitema 

All evenings except Wednesday are available for small group discussions 
and work on the seminar project. 



Tuesday . March 29 



Presiding 

Test Validity in Predictive 
Research 

Discussion of Dr. Forehand's Paper 
Presiding 

Available Tests and Their Use in, 
Research- in Vocational Education 



Randall Nelson 
Garlie Forehand 



John Flanagan 
Margaret . Crawford 



Opportunity for individuals to consult with Dr. Forehand, Dr 
work On the seminar project. 




yi 



Crawford, or 
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3:00 

6:30 



8: 30 Presiding 
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Wednesday , March 30 



Test Validity in Experimental 
and Curriculum Research 



Irving Cross 
Robert Stake 



Discussion of Dr. Stake's Paper 

Tour of National Bureau of Standards, Boulder, Colorado 
Banquet Jim Wilson, Presiding 

Program Robert tt-own 



I 0 
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i , 





Thursday , March 31 



8:30 


Presiding 


Ralph Canada 


10:30 


Validity in Survey Research 
Discussion of Dr. Knox's Paper 


Alan Knox 



Friday . April 1 



8:30 


Presiding 


Joe Roberts 




Evaluation Activity 


Charles W. Hill 
Douglas Sjogren 


9:00 


Decisions in selecting and 
developing instruments for a 


John Coster 
Tom Lyons 




research project 


10:30 


Summary of Workshop 


Charles 0. Neidt 
C. Dean Miller 


11:30 


Adjournment 





Personnel Appearing on Program 

J. Stanley Ahmann, Academic Vice President, Colorado State University 

Juanita Roberts, Instructor, Vocational Education, Colorado State University 

Herbert Benson, Professor and Head, Vocational Education, Colorado State University 

Charles 0. Neidt, Professor and Head, Department of Psychology, Colorado State 

University ^ 

Charles W. Hill, Professor of Agricultural Education, Cornell University and 

Seminar Project Director 

C. Dean Miller, Assistant Professor, Department of Psychology, Colorado State 

Ifoiversity 
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Garlie Forehand, Professor and Acting Head, Department of Psychology, Carnegie 

Institute of Technology 

Randall Nelson, Assistant Professor, Department of Education, Colorado State 

University 

John Flanagan, Assistant Professor, Department of Education, Colorado State 

University 

Margaret Crawford, Assistant Dean of Counseling and Guidance, Los Angeles Trade 

Technical College 

Irving Cross, Assistant Professor, Vocational Education, Colorado State University 

Robert E. Stake, Associate Professor, Center for Instructional Research and 

Curriculum Evaluation, University of Illinois 

Jim Wilson, Assistant Professor, Vocational Education, Colorado State University 

Robert Brown, Denver Public Schools 

Ralph Canada, Professor, Vocational Education, Colorado State University 

Alan Knox, Professor, Department of Education, Teachers College, Columbia University 

Joe Roberts, Instructor, Vocational Education, Colorado State University 

John Coster, Professor, Agricultural Education, University of Nebraska 

Tom Lyons , Project Director, Agricultural Education, University of Nebraska 

Douglas Sjogren, Associate Professor, Vocational Education, Colorado State University 

and Seminar Director 

David Sheppard, Research Associate, Vocational Education, Colorado State University 
Charles Stew ar t , Graduate Assistant , Colorado State University 
James Sheard, Graduate Assistant, Colorado State University 
Bradley Huitema, Graduate Assistant, Colorado State University 

Program and Planning Committee 

Douglas Sjogren, Chairman and Seminar Director 

Charles 0. Neidt C. Dean Miller 

Ralph Canada John Flanagan 

Joe Roberts Juanita Roberts 



Participants 



Gladys E. Abad 
Richard D. Ashmun 
Robert P. Barnes 
David Bj or quiet 
James Eugene Bottoms 
James E. Bowman 
V. R. Cardozier 
Hester Chadderdon 
Virgil E. Christensen 
John K. Coster 
Harold R. Cushman 
Lawrence W. Drabick 
John H. ERickson 
B. B. Griffith 
J. Marshall Hanna 
Hildegarde Johnson 
Denis Kigin 
Ronald L. Koble 
Herschel T. Lester, Jr. 
Nell P. Logan 
Gordon G. McMahon 



Wilbur R. Miller 
William E. Mortimer 
Robert Mullen 
Orville W. Nelson 
A. Laveme Phillips 
Lloyd J. Phipps 
Alan G. Robertson 
Charles H. Rogers 
William John Schill 
William P. Spence 
George Storm 
Alan R. Suess 
H. Paul Sweany 
Erroll John Terrell 
0. E. Thompson 
Bruce W. Tuckman 
Neal E. Vivian 
Richard H. Wilson 
Fred Winger 
Lawrence S. Wright 

Ottc Pr Legg 
Richard Otte 
Robert Knoebel 
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National Vocational Education Research Seminar 

on 

OCCUPATIONAL MOBILITY AND MIGRATION 
North Carolina State University 
April 18-22, 1966 

♦ 

Harry Beard, Seminar Director 



PROGRAM 



Objectives 

1. To secure a better understanding of the problems and decisions facing educational 
policymakers, program planners, administrators and teachers. 

2. To develop a conceptual framework for studying socio-economic mobility. 

3. To develop an understanding of the status of research in labor supply, labor 
demand, migration, and socio-economic mobility. 

4. To become acquainted with appropriate research rationales, designs and 
methodologies . 

5. To identify problem areas for research. 

6. To identify specific researchable problems. 

7. To acquire an understanding of the resources available, particularly at the 
state level, to conduct research in occupational mobility and migration. 

8. To begin to develop lines of communication to exploit available resources to 
conduct research and to disseminate research findings. 



Sunday Evening , April 17 

6: 00-9; 00 Registration 



Monday , April 18 

Chairman: H. G. Beard, Associate Professor, Education 

and Rural Sociology, North Carolina State University 

9:00 Greetings Selz C. Mayo, Head, 

Sociology and Anthropology and Rural 
Sociology 

North Carolina State University 
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9:15 


Introduction of Participants 




9:30 


Overview of the Seminar 




9:45 


Pre-seminar Evaluation 


C. W. Hill 

Project Director of Research Seminars 
Cornell University 


10:30 


The Need for Research and Its 


Dean Rupert N. Evans 




Utilization in Local, Regional 


College of Education 




and State Systems of Education 


University of Illinois 


11:30 


Reaction 


H. M. Hamlin, Visiting Professor 

School of Education 

North Carolina State University 


11:45 


Discussion 




1:45 


Setting the Stage for a Study 


E. Walton Jones 




of Occupational Mobility and 


Associate Professor of Economics 




Migration 


North Carolina State University 


2:30 


Discussion 




3:30-5:00 


Organizing Interest Groups (at least four) 





Tuesday, 


April 19 




Chairman: H, 


. M. Hamlin 


8:30 


Manpower Supply in the 


Howard Rosen 




United States 


U.S. Office of Manpower Policy 
Evaluation and Research, 

U.S. Department of Labor 


9:10 


Reaction 


C. E. Bishop, Head 
Department of Economics 
North Carolina State University 


9:25 


Discussion 




10:30 


Manpower Requirements by 


Sol Swerdloff 




Industry and Occupation 


Bureau of Labor Statistics 
U.S. Department of Labor 


11:10 


Reaction 


J. G. Maddox, Professor 
Department of Economics 






North Carolina State University 


11:25 


Discussion 




1:30 


The Role of Spatial Mobility in 


Everett S. Lee, Professor 




Occupational Change 


Population Studies Center 
University of Pennsylvania 


>: 10-2: 25 


Reaction 


C. Horace Hamilton, Professor 



Department of Rural Sociology 
North Carolina State University 



2:25-3:00 Discussion 



3? 30 The Social-Psychological 
Dimensions of Occupational 
Mobility 

4 : 10 Reaction 



4:25 Discussion 
7:00-9:00 Individual 



William P. Kuvlesky, Assistant Professor 
Department of Agricultural Economics 
and Sociology 
Texas A & M University 

L. W. Drfbick 

Research Associate Professor 
Departments of Agricultural Education 
and Rural Sociology 
North Carolina State University 



Conferences by Appointment with Consultants 



Wednesday , April 20 

Chairman: Charles E. Lewis, Administrative Assistant, 

Center for Research, Development and Training, 

North Carolina State University 

8:30 Understanding Social Mobility Selz C. Mayo, Head, 

Department of Sociology 
North Carolina State University 

9:00 Reaction 

Chairman of Interest Groups 
9:30 Discussion 

10:30 Planning Work of Interest Groups 

11:00 Meetings of Interest Groups ( consultants available) 

1:30-5:00 Meetings of Interest Groups (consultants available) 

7 : 00 Banquet 



Address 



He F. Robinson 

Administrative Dean for Research 
North Carolina State University 



Thursday , April 21 

8:30-12:00 Meetings of Interest Groups (consultants available) 
• 1:30 Report from Interest Group No. 1 

2:15 Report from interest Group No. 2 

3:30 Report from Interest Group No. 3 

4:15 Report from Interest Group No. 4 
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Friday , April 22 



8:30 


Panel: Opportunities Ahead. 


Representatives of Interest Groups 


10:30 


Policy Decisions Facing State 
Boards of Education and the Need 
for Research 


Dallas Herring, Chairman, 

North Carolina State Board of Education 


11:10 


Reaction 


A. G. Bullard 

Director of Vocational Education 
North Carolina Department of Public 
Instruction 


11:25 


Post-seminar Evaluation 


C. W. Hill, Project Director 


12:00 


Adjournment 





Seminar Planning Committee 

H. M. Hamlin, Visiting Professor, School of Education, North Carolina State University 

C. Horace Hamilton, Professor, Department of Rural Sociology, North Carolina State 

University 

E. Walton Jones, Associate Professor, Department of Economics, North Carolina State 

University, Vice Chairman 

Charles E. Lewis, Administrative Assistant, Center for Research, Development and 

Training in Occupational Education, North Carolina State University 

Harry G. Beard, Associate Professor, Departments of Education and Rural Sociology, 

North Carolina State University, Chairman, Planning Committee 



Participants 



John M. Adams, Jr. 
Arch B. Alexander 
Philip G. Baird 
H. E. Beam 
Irene Beavers 
James W. Bechet 
8. T. Brantner 
V. E. Burgener 
Calfrey C. Calhoun 
Harley R. Cheshire 
Angel L. Alicea-Colon 
Charlie M. Curtis 
Jeanne Dost 
John Ephraim 
John C. Gilliam 
Nathan Gross 
Leslie V. Hawkins 



Thomas A. Hoerner 
William A. Householder 
James Kiefert 
L. C. McDowell 
0. Donald Meaders 
Carl A. Moeller 
Lloyd A. Ponder 
Jane Preston 
A1 Ringo 

William B. Runge 
Robert E. Scott 
Benjamin Clifford Tinnell 
Douglas C. Towne 
Mercedes I. Vercher 
Ralph J. Woodin 

Lawrence Braaten 
Emanuel Weinstein ' 
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National Vocational Education Research Seminar 

on 

RESEARCH DESIGN 
Cornell University- 
May 2-6, 1966 

Frederick K.T.Tom, Seminar Director 




PROGRAM 




Over-all Purpose of the Seminar 

This seminar was designed to develop in the participants a greater interest in 
conducting experimental research of increasingly higher quality and quantity in the 
field of vocational education and to improve their ability to design such research. 



Specific Objectives 



' As a result of attending the seminar, it is hoped that the participants would 
be able to: 

1. Demonstrate an increased interest in conducting experimental research 
in vocational education. 

2. Identify the principles and procedures which should be followed in 
designing experimental research. 

3. Select the proper statistical tools. 

4. Determine whether a given sampling procedure is defensible for a given 
piece of research. 

5. Recognize the strengths and weaknesses of selected research proposals. 

6. Cite the research functions of the U.S. Office of Education under 
Public Law 88-210, Section 4(c). 



Monday , May 2 

8:30 Registraticn 

9:00 Greetings and Welcome Nyle C. Brady, Director of Research 

New York State College of Agriculture 
Cornell University 




ppnpppp 
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9:15 



10:30 

1:15 

3:15 

4:30 

9:00 

10:30 

11:45 

1:15 

3:15 

6:30 



9:00 

10:30 



Overview and Plan- of Seminar Frederick K.T.Tom, Seminar Director 

Associate Professor of Agricultural 
Education 

’ Cornell University 



Development of the National 
Vocational Education Research 
Seminars 



Charles W. Hill,, Project Director 
Professor of Agricultural Education 
Cornell University 



Keynote Address - The Role of 
Experimental Research in 
Education 

Some Perspectives on the 
Research Process 



More on the Research Process and 
Its Implications for Proposal 
Writing 

Tour of Campus 



Jason Millman, Associate Professor of 
Educational Psychology and Measurement 
Cornell University 

David R. Krathwohl, Dean, 

School of Education 
Syracuse University 




Dean Krathwohl 



I 



r. . 1 



; 

i 



Tuesday . May 3 



The Preparation of Research 
Proposals and the Design of 
Experimental Research 

The Arithmetic of Analysis of 
Variance 



Lunch (Statler) 

The Preparation of Research 
Proposals and the Design of 
Experimental Research 

Topics in the Analysis of 
Variance 

Banquet at Statler, Faculty Lounge 
Greetings 



Dean Krathwohl 



Donald G. MacEachera 
Assistant Professor of Educational 
Psychology, College of Education 
University of Minnesota 



Dean Krathwohl 



Professor MacEachern 



Frederick H. Stutz, Dean, 
School of Education 
Cornell University 



Wednesday . May 4 

The Analysis of Variance in Professor MacEachern 

Research 

Sampling Procedures in Research Professor MacEachern 







Small group sessions 



Is 15 Study of Four Selected Research 

Proposals in Vocational Education 



3:15 Study of Four Selected . . . 
(continued) 



4:00 An Analysis of Above Selected 
Research Proposals 



Eunice H. Jones, Project Coordinator, 
Human Resources Branch, USOE 



Virginia F. Thomas, Specialist, 
Educational Resources Development 



Branch, USOE 



Jack A. Wilson, Labor Economist 
Employment Opportunities Branch, USOE 



Sherrill McMillen, Director, 
Planning and Program Development 



Branch, USOE 



Thursday , May 5 



9:00 Current Experimental Studies in Vocational Education 

1. An Experimental Evaluation of Approaches to Preparing High School 
Students for Agriculture Occupations Other than Farming - 

John K. Coster, Professor and Director of Research, College of 
Agriculture and Home Economics, University of Nebraska 

2. Maximization of the Professional Potential of Home Economics Teachers 
through Group Counseling - Elizabeth M. Ray, Associate Professor of 
Home Economics Education, Pennsylvania State University 

10:30 3. The Relative Effectiveness of the Traditional and Two Modified 



Methods of Organizing Information Sheets - David J. Pucel, Instructor, 
Department of Industrial Education, University of Minnesota 

4. Development of a Curriculum and Materials for Teaching Basic 
Vocational Talents - John T. Dailey, Research Professor, George 
Washington University 



1:15 Current Research Developments in 
the U.S. Office of Education 



Sidney C. High 
Specialist in Research 



3:15 Latest Developments in Computer 



Karl L. Zinn, Research Associate 
Center for Research on Learning and 



Science with Implications for 
the Educational Researcher 



Teaching 

University of Michigan 



Friday , May 6 



9:00 Formal Review of Research 



Sidney High, USOE, Chairman 
Donald MacEachem, Minnesota 
Sara Blackwell, Cornell 
Harold R. Cushman , Cornell 
D. Bob Gowin, Cornell 



Proposals Submitted by Seminar 
Participants (by role-playing 
panel) 



Formal Review (continued) 
Evaluation of Seminar 



Professor Hill 



10:30 

11:30 



Members of the Seminar Advisory Committee 

Frederick H. Stutz Dean, School of Education 

Harold R. Cushman Professor of Agricultural Education 

Felician F. Foltman Professor of Industrial & Labor Relations 

Jason Millman Associate Professor of Educational Psychology and Measurement 

Helen Y. Nelson Associate Professor of Home Economics Education 

Frederick K.T.Tom Associate Professor of Agricultural Education, Chairman and 

Seminar Director 



Participants 



Ray Agan 
Zelpha Bates 
James Bikkie 
Ilene Brown 
Charles Bryant 
Earl T. Carpenter 
Kenneth R. Clay 
James Clouse 
Wayne E. Courtney 
David G. Craig 
Jean Cooper 
Julia I. Dalrymple 
John Egermeier 
Roswell E. Fairbank 
Everett R. Glazener 
Kenneth L. Hansen 
E. Edward Harris 
Alberta D. Hill 
William L. Hull 
Denver B. Hutson 



Arthur K. Jensen 
John W. Karnes 
Carmela C. Kingston 
Shirley M. Kreutz 
Milton E. Larson 
Calvin D. Lowe 
Roy E. McDermott 
Charles W. Nichols 
William R. Pasewark 
Bemadine H. Peterson 
Robert Poland 
Dale J. Prediger 
Kenneth L. Rowe 
Twyla Shear 
John F. Stephens 
Ruth Stovall 
Helen W. Taylor 
Robert M. Tomlinson 
Darrell S. Willey 

Eunice H. Jones 
Sherrill McMillen 
Withro McEnge 
Virginia F. Thomas 
Jack A. Wilson 



National Vocational Education Research Seminar 



✓ 



on 

CURRICULUM EVALUATION 
University of Illinois 
May 16-20, 1966 

William Schill, Seminar Director 






i 



PROGRAM 



Monday , May l6 



Registration 

Tour of Socrates Installation - conducted by Henry Lippert 

Bureau of Educational Research 
University of Illinois 



8:00 

9:00 
& 

11:00 
12 : 00 Welcome 

Presentations 
2:00 Chairman 

Problems and Prospects in 
Vocational Education 

Responses: 



Tuesday 

8: 30 Chairman 

Sociology and Curriculum 
Responses : 



Rupert Evans, Dean, 

College of Education 
University of Illinois 

Charles W. Hill, Project Director 

M. Ray Karnes, College of Education 
University of Illinois 

Harry Broudy, College of Education 
University of Illinois 

Stephanie Edgerton, College of Education 
University of Wisconsin 

James Gallagher, College of Education 
University of Illinois 

Jacob Stem, College of Education 
University of Illinois 



May 17 

Robert Tinkham 
University of Illinois 

David Street, Department of Sociology 
University of Chicago 

Walter Franke, Illinois 
William J. Schill, Illinois 
Rutherford Lockette, Trenton State College 
Trenton, New Jersey 



****** 
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Chairman 

Criterion Problems and 
Curriculum Evaluation 

Responses: 



Alfred Krebs 
University of Illinois 

Donald Leton 

Bureau of Educational Research 
University of Hawaii 

Harry Anderson, College of Education, 
University of Georgia 

Ralph Mason, School of Business 
Indiana State University 

Robert Wasson, College of Education 
University of Iowa 



Wednesday . May 18 



Chairman 

Psychological Aspect of 
Curriculum Evaluation 

Responses : 



Chairman 

The Countenance of Educational 
Evaluation 

Responses : 



Robert Campbell 
University of Illinois 

David Ausubel, College of Education 
University of Illinois 

Robert Tomlinson, College of Education 
University of Illinois 

Richard Salinger 
University of Illinois 

Barbara Rosenquist 

System Development Corporation 

Santa Monica, California 

Jacob Stem, College of Education 
University of Illinois 

Robert Stake, College of Education 
Univwrsity of Illinois 

J. Thomas Hastings 

Bureau of Educational Research 

University of Illinois 

Philip Tiemann 

Office of Instructional Resources 
University of Illinois 

Robert A. Campbell, College of Education 
University of Illinois 



Thursday . May 19 

Robert Tomlinson, College of Education 
University of Illinois 

John Easley, Jr. College of Education 
University of Illinois 



Chairman 

Evaluation Problems of the 
UICSM Curriculum Project 



9 6 



Responses: 

1:30 Chairman 

Administrative Aspects of 
Curriculum Evaluation 

Responses : 



Richard Spencer 

Office of Instructional Resources 
University of Illinois 

Jerry Dobrovolny 

Department of General Engineering 
University of Illinois 

James Lipham, School of Education 
University of Wisconsin 

Arthur McBeath 
University of Illinois 



Friday . May 20 

9 s 00 Chairman William J. Schill 

University of Illinois 

Seminar summary and a charge to the participants 
10:00 Seminar evaluation Charles W. Hill, Project Director 



Participants 



Margaret V. Barkley 
Walter A. Bialobrzeski 
Harold M. Byram 
Edgar E. Clanin 
Aleene Cross 
Gene L. Dahlin 
William E. Drake 
Charles C. Drawbaugh 
Vivien K. Ely 
Nevin R. Frantz 
Donald G. Green 
James W. Hensel 
James H. Hutchinson 
Joseph T. Irapelletteri 
Mildred B. Johnson 
Allen Kelsey 
Elizabeth E. Kerr 
Earl E. Knebel 
Louise Lemon 



Rutherford E. Lockette 
Phyllis K. Lowe 
W. Howard Martin 
Ralph E. Mason 
John D. Mattingly 
Clarence Maze, Jr. 
Marie P. Meyer 
Warren G. Meyer 
Jerome Moss, Jr. 

Helen Nelson 
Wilmot F. Oliver 
Elizabeth Ray 
Agnes F. Ridley 
Lucy Robinson 
John H. Rodgers 
Howard Rosenwinkel 
Glenn Z. Stevens 
Ben S. Vineyard 

Mary Lee Hurt 
Earl M. Bowler 
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